♻️ API to run evaluations of the FAIR principles (Findable, Accessible, Interoperable, Reusable) on online resources

Overview

♻️ FAIR enough 🎯

Run backend tests

An OpenAPI where anyone can run evaluations to assess how compliant to the FAIR principles is a resource, given the resource identifier (URI/URL).

Using FastAPI, Pydantic and MongoDB

📥️ Requirements

  • Docker

  • Docker Compose

  • Poetry if you need to install new Python packages.

  • Node.js (with npm) if you need to do frontend development

📝 Add an assessment

All assessments used to run evaluations are python scripts defined in the same folder: https://github.com/MaastrichtU-IDS/fair-enough/tree/main/backend/app/assessments

Feel free to add new assessments and send a pull request! To create a new assessment:

  • Optionally, create a folder if you want to group multiple assessments under a same category

  • Copy an existing assessment to get started

  • Change the attributes of this assessment to describe it so that users can easily understand what your assessment do. Provide your email in the author attribute.

  • Add the code in the evaluate() function, 2 variables are passed to the assessment, plus you can access the assessment object itself to log what the test is trying to do, and why it success or fail:

    • eval: evaluation object that you can use to pass data between assessments (e.g. to pass the license URL, or JSON-LD metadata your assessment retrieves)

    • g: RDFLib graph of the RDF retrieved when searching for the resource metadata

    • self: the assessment object itself, can be used to perform various logging actions related to the test (don't use print otherwise it will not show up in the evaluation results returned by the API)

      self.log('This print a regular event', '✔️') # 2nd arg (prefix added to the log) is optional
      self.success('This will also increase the score of the assessment by 1')
      self.bonus('This will also increase the bonus score of the assessment by 1')
      self.error('This will print a failure while running the assessment')

Most of the Python code for the API is in https://github.com/MaastrichtU-IDS/fair-enough/tree/main/backend/app

🐳 Backend local development

Start the stack for development locally with Docker Compose from the root folder of this repository:

docker-compose up -d

Now you can open your browser and interact with these URLs:

To check the logs of a specific service, run:

docker-compose logs backend

To delete the volume and reset the database, run:

docker-compose down
docker volume rm fair-enough_mongodb-data

You can also run this script to reset the database, and restart the docker-compose:

./reset_local_db.sh

If you need to completely reset the Python cache:

docker-compose down
sudo rm -rf **/__pycache__
docker-compose build --no-cache

General workflow

By default, the dependencies are managed with Poetry, go there and install it.

From ./backend/ you can install all the dependencies with:

poetry install

To add new dependencies, run:

poetry add my-package

If you install a new package you will need to stop the current docker-compose running, then restarting it to rebuild the docker image:

docker-compose up --build --force-recreate

You can start a shell session with the new environment with:

poetry shell

Next, open your editor at ./backend/ (instead of the project root: ./), so that you see an ./app/ directory with your code inside. That way, your editor will be able to find all the imports, etc. Make sure your editor uses the environment you just created with Poetry.

Docker Compose Override

During development, you can change Docker Compose settings that will only affect the local development environment, in the file docker-compose.override.yml

docker-compose up -d

To get inside the container with a bash session you can exec inside the running container:

docker-compose exec backend bash

Backend tests

Test running stack

If your stack is already up and you just want to run the tests, you can use:

docker-compose exec backend /app/tests-start.sh

That /app/tests-start.sh script just calls pytest after making sure that the rest of the stack is running. If you need to pass extra arguments to pytest, you can pass them to that command and they will be forwarded.

For example, to stop on first error:

docker-compose exec backend bash /app/tests-start.sh -x

Test new stack

To test the backend run:

DOMAIN=backend sh ./scripts/test.sh

The file ./scripts/test.sh has the commands to generate a testing docker-stack.yml file, start the stack and test it.

Local tests

Start the stack with this command:

DOMAIN=backend sh ./scripts/test-local.sh

The ./backend directory is mounted as a "host volume" inside the docker container (set in the file docker-compose.dev.volumes.yml). You can rerun the test on live code:

docker-compose exec backend /app/tests-start.sh

Test Coverage

Because the test scripts forward arguments to pytest, you can enable test coverage HTML report generation by passing --cov-report=html.

To run the local tests with coverage HTML reports:

DOMAIN=backend sh ./scripts/test-local.sh --cov-report=html

To run the tests in a running stack with coverage HTML reports:

docker-compose exec backend bash /app/tests-start.sh --cov-report=html

🖥️ Frontend development

  • Enter the frontend directory, install the NPM packages and start the live server using the npm scripts:
cd frontend
npm install
npm run serve

Then open your browser at http://localhost:8080

If you have Vue CLI installed, you can also run vue ui to control, configure, serve, and analyze your application using a nice local web user interface.

🚀 Deployment

Traefik network

This stack expects the public Traefik network to be named traefik-public, just as in the tutorials in DockerSwarm.rocks.

If you need to use a different Traefik public network name, update it in the docker-compose.yml files, in the section:

networks:
  traefik-public:
    external: true

Change traefik-public to the name of the used Traefik network. And then update it in the file .env:

TRAEFIK_PUBLIC_NETWORK=traefik-public

Docker Compose files and env vars

There is a main docker-compose.yml file with all the configurations that apply to the whole stack, it is used automatically by docker-compose.

And there's also a docker-compose.override.yml with overrides for development, for example to mount the source code as a volume. It is used automatically by docker-compose to apply overrides on top of docker-compose.yml.

These Docker Compose files use the .env file containing configurations to be injected as environment variables in the containers.

They also use some additional configurations taken from environment variables set in the scripts before calling the docker-compose command.

It is all designed to support several "stages", like development, building, testing, and deployment. Also, allowing the deployment to different environments like staging and production (and you can add more environments very easily).

They are designed to have the minimum repetition of code and configurations, so that if you need to change something, you have to change it in the minimum amount of places. That's why files use environment variables that get auto-expanded. That way, if for example, you want to use a different domain, you can call the docker-compose command with a different DOMAIN environment variable instead of having to change the domain in several places inside the Docker Compose files.

Also, if you want to have another deployment environment, say preprod, you just have to change environment variables, but you can keep using the same Docker Compose files.

🔗 Links

Livestream logs:

Project bootstrapped with https://github.com/tiangolo/full-stack-fastapi-postgresql

Comments
  • not detecting multiple license terms

    not detecting multiple license terms

    Evaluation on Uniprot entry: https://fair-enough.semanticscience.org/evaluation/619df87cc671558d15c297d4

    fails to recognize dcterms:license in metadata

    bug 
    opened by micheldumontier 2
  • Better querying of the evaluations results

    Better querying of the evaluations results

    Users should be able to query the evaluations results to answer questions like "get me all evaluations that have a score of 1 for the metric test i1-structrured metadata and a score of 0 for i1-license"

    Either:

    • add functions to graphql, cf. https://dgraph.io/docs/graphql/schema/search/
    • Or load the whole DB in a triplestore for SPARQL querying
    opened by vemonet 1
  • Sort Metrics tests and Collections in categories

    Sort Metrics tests and Collections in categories

    We might want to have a multi dimension categorization for collections and metrics tests, e.g.

    domains = [ 'biomedical', 'geography']
    resource_type= ['publication', 'ontology', 'tabular data']
    

    That will help users to know which collection they should use when evaluating specific types of data.

    And help them to choose the right metrics tests when composing a collection.

    @rcelebi

    opened by vemonet 1
  • issue in starting containers

    issue in starting containers

    checkout with github dektop, using ubuntu shell. needed to chmod 755 bash scripts:

    chmod 755 init_metric_tests.sh
    chmod 755 backend/prestart.sh
    

    and also clean them

    sed -i -e 's/\r$//' init_metric_tests.sh
    sed -i -e 's/\r$//' backend/prestart.sh
    
    
    opened by micheldumontier 0
  • Local development support for HTTPS + ORCID

    Local development support for HTTPS + ORCID

    ORCID authentication requires HTTPS redirects. Out of the box, the application does not respond to HTTPS addresses on localhost. How to configure this?

    help wanted 
    opened by micheldumontier 1
  • Create a Web Extension to evaluate researchers publications from aggregators

    Create a Web Extension to evaluate researchers publications from aggregators

    This extension could be called by a researcher when on his Pure, or Google Scholar page. It will automatically extract the URI of each publication of the researcher, and run a FAIR evaluation for each publication

    Generate a report on the FAIRness of all publications of the researcher

    We should use the new WebExtension standard to build an extension that will work on all browsers (at least Firefox and Chrome, safari is dead anyway)

    @pedrohserrano

    good first issue help wanted 
    opened by vemonet 0
  • create

    create "resource-centric" view

    For a given resource, list all the tests performed and the scores, from newest to oldest. make infographic to visualise the scores across time add button to "test now"

    good first issue help wanted 
    opened by micheldumontier 3
  • Map base and bonus scores to F,A,I,R

    Map base and bonus scores to F,A,I,R

    It's difficult to understand the breakdown of the base and bonus scores to each of F,A,I, R. might suggest some kind of interactive chart that i) shows where the resource is passing/failing, and ii) is clickable to the specific assessment section.

    enhancement 
    opened by micheldumontier 2
  • enable text search on all metadata

    enable text search on all metadata

    I'd like to be able to search all the evaluations using a global search input box. for instance, to search for "sea ice" and get matching records.

    enhancement 
    opened by micheldumontier 4
Releases(0.0.1)
  • 0.0.1(Jan 27, 2022)

    FAIR enough release before using FAIRMetrics specifications and tests through APIs

    Using Celery workers and RabbitMQ to execute the tests

    Source code(tar.gz)
    Source code(zip)
Owner
Maastricht University IDS
Institute of Data Science at Maastricht University
Maastricht University IDS
Converts a text file of songs to a playlist on your Spotify account.

Playlist Converter Convert a text file of songs to a playlist on your Spotify account. Create your playlists faster instead of manually searching for

Priya Aggarwal 18 Dec 21, 2022
A python telegram bot to fetch the details of an ipadress with help of ip-api

ipfetcher A python(Pyrogram) oriented telegram bot to fetch the details of an ipadress developed by @riz4d with the API of https://ip-api.com Deployme

Mohamed Rizad 5 Mar 12, 2022
A code to match you with the perfect Taylor Swift song for your mood and relationship status.

taylorswift A package for matching your current mood and relationship status to a suitable Taylor Swift song. Requirements: Python 2 or 3, and the num

Megan Mansfield 82 Dec 09, 2022
MemeBot - A discord bot that tracks how good people's memes are

MemeBot A discord Meme "Karma" Tracking bot Dependancies Make sure you have pymongo installed and a mongodb cluster setup with two collections. pip in

Uday Sharma 3 Aug 10, 2022
🚀 An asynchronous python API wrapper meant to replace discord.py - Snappy discord api wrapper written with aiohttp & websockets

Pincer An asynchronous python API wrapper meant to replace discord.py ❗ The package is currently within the planning phase 📌 Links |Join the discord

Pincer 125 Dec 26, 2022
Aqui está disponível GRATUITAMENTE, um bot de discord feito em python, saiba que, terá que criar seu bot como aplicação, e utilizar seu próprio token, e lembrando, é um bot básico, não se utiliza Cogs nem slash commands nele!

BotDiscordPython Aqui está disponível GRATUITAMENTE, um bot de discord feito em python, saiba que, terá que criar seu bot como aplicação, e utilizar s

Matheus Muguet 4 Feb 05, 2022
Randomly selects two teams based on who is in a voice channel on Discord

TeamPickerDiscordBot Randomly selects two teams based on who is in a voice channel on Discord What I Learned The ins and outs of Python as this was my

Brecken Enneking 2 Jan 27, 2022
A discord bot to check if messages have the correct code formatting.

discord-code-formatter A discord bot to check if messages have the correct code formatting. This was a basic project to help me learn Python and learn

Nash Boisvert 1 Nov 23, 2021
Pls give vaccine.

Pls Give Vaccine A script to spam yourself with vaccine notifications. Explore the docs » View Demo · Report Bug · Request Feature Table of Contents A

Rohan Mukherjee 3 Oct 27, 2021
Python client for using Prefect Cloud with Saturn Cloud

prefect-saturn prefect-saturn is a Python package that makes it easy to run Prefect Cloud flows on a Dask cluster with Saturn Cloud. For a detailed tu

Saturn Cloud 15 Dec 07, 2022
Facebook open graph api implementation using the Django web framework in python

Django Facebook by Thierry Schellenbach (mellowmorning.com) Status Django and Facebook are both rapidly changing at the moment. Meanwhile, I'm caught

Thierry Schellenbach 1.4k Dec 29, 2022
Python library to interact with a Z-Wave JS server.

zwave-js-server-python Python library for communicating with zwave-js-server. Goal for this library is to replicate the structure and the events of Z-

Home Assistant Libraries 54 Dec 18, 2022
Telegram Bot for generating and decoding QR-codes

Telegram openqrgen_bot Telegram Bot that generates from user's messages and decodes QR-codes from photos. Also contains rickroll detection :) Just typ

2 Nov 14, 2021
High-Resolution Differential Z-Belt Mod for V0 (with optional Kirigami support)

V0-DBM This is a high-resolution differential pulley system belt mod for the Z-axis on Voron 0 with optional Kirigami Bed support. NOTE: Alpha version

Simon Küppers 11 Jan 07, 2023
A python package to easy the integration with Direct Online Pay (Mpesa, TigoPesa, AirtelMoney, Card Payments)

A python package to easy the integration with Direct Online Pay (DPO) which easily allow you easily integrate with payment options once without having to deal with each of them individually;

Jordan Kalebu 2 Nov 25, 2021
Auto-Approved-Bot - Auto Approved Invaite Link Request Telegram Bot

🤖 𝗔𝘂𝘁𝗼-𝗔𝗽𝗽𝗿𝗼𝘃𝗲-𝗕𝗼𝘁 🤖 ℹ️ 𝗨𝘀𝗲𝗴𝗲 ℹ️ When a join request invita

Muhammed 32 Dec 18, 2022
Python Discord Server Nuker

Untitled Nuker Python Discord Server Nuker Features: Ban Everyone Kick Everyone Rename Everyone Spam To All Channels Delete All Channels Delete All Ro

22 Dec 22, 2022
Add members to unlimited telegram channels and groups

Program Features 📌 Coded with Python version 10. 📌 without the need for a proxy. 📌 without the need for a Telegram ID. 📌 Ability to add infinite p

hack4lx 10 Nov 25, 2022
Sail is a free CLI tool to deploy, manage and scale WordPress applications in the DigitalOcean cloud.

Deploy WordPress to DigitalOcean with Sail Sail is a free CLI tool to deploy, manage and scale WordPress applications in the DigitalOcean cloud. Conte

Konstantin Kovshenin 159 Dec 12, 2022
Bot that embeds a random hysterical meme from Reddit into your text channel as an embedded message, using an API call.

Discord_Meme_Bot 🤣 Bot that embeds a random hysterical meme from Reddit into your text channel as an embedded message, using an API call. Add the bot

2 Jan 16, 2022