MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

Overview

MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

PyPI version test Docker Cloud Build Status Documentation Status Maintainability codecov MIT

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Documentation: https://minerva-ui.readthedocs.io

Chat: Gitter

key features

All You Need Is Dataset

MINERVA only requires datasets to start offline deep reinforcement learning. Any combinations of vector observations and image observations with discrete actions and continuous actions are supported.

🔰 Stunning GUI

MINERVA provides designed with intuitive GUI to let everyone lerverage extremely powerful algorithms without barriers. The GUI is developed as a Single Page Application (SPA) to make it work in the eye-opening speed.

🚀 Powerful Algorithm

MINERVA is powered by d3rlpy, a powerful offline deep reinforcement learning library for Python, to provide extremely powerful algorithms in an out-of-the-box way. The trained policy can be exported as TorchScript and ONNX.

installation

PyPI

$ pip install minerva-ui

Docker

$ docker run -d --gpus all -p 9000:9000 --name minerva takuseno/minerva:latest

update guide

If you update MINERVA, the database schema should be also updated as follows:

$ pip install -U minerva-ui
$ minerva upgrade-db

usage

run server

At the first time, ~/.minerva will be automatically created to store database, uploaded datasets and training metrics.

$ minerva run

By default, you can access to MINERVA interface at http://localhost:9000 . You can change the host and port with --host and --port arguments respectively.

delete data

You can delete entire data (~/.minerva) as follows:

$ minerva clean

contributions

build

$ npm install
$ npm run build

coding style

This repository is fully formatted with yapf and standard. You can format the entire scripts as follows:

$ ./scripts/format

lint

This repository is fully analyzed with Pylint, ESLint and sass-lint. You can run analysis as follows:

$ ./scripts/lint

test

The unit tests are provided as much as possible. This repository is using pytest-cov instead of pytest. You can run the entire tests as follows:

$ ./scripts/test

acknowledgement

This work is supported by Information-technology Promotion Agency, Japan (IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal year 2020.

The concept of the GUI software for deep reinforcement learning is inspired by DeepAnalyzer from Ghelia inc. I'm showing the great respect to the team here.

Comments
  • Bump y18n from 4.0.0 to 4.0.3

    Bump y18n from 4.0.0 to 4.0.3

    Bumps y18n from 4.0.0 to 4.0.3.

    Changelog

    Sourced from y18n's changelog.

    4.0.3 (2021-04-07)

    Bug Fixes

    • release: 4.x.x should not enforce Node 10 (#126) (1e21a53)

    4.0.1 (2020-11-30)

    Bug Fixes

    Commits
    Maintainer changes

    This version was pushed to npm by oss-bot, a new releaser for y18n since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump pillow from 7.1.2 to 8.1.1

    Bump pillow from 7.1.2 to 8.1.1

    Bumps pillow from 7.1.2 to 8.1.1.

    Release notes

    Sourced from pillow's releases.

    8.1.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

    8.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.1.1 (2021-03-01)

    • Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

    • Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

    • Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

    • Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

    • Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

    • PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

    8.1.0 (2021-01-02)

    • Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

    • Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

    • Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

    • Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

    • Makefile updates #5159 [wiredfool, radarhere]

    • Add support for PySide6 #5161 [hugovk]

    • Use disposal settings from previous frame in APNG #5126 [radarhere]

    • Added exception explaining that repr_png saves to PNG #5139 [radarhere]

    • Use previous disposal method in GIF load_end #5125 [radarhere]

    ... (truncated)

    Commits
    • 741d874 8.1.1 version bump
    • 179cd1c Added 8.1.1 release notes to index
    • 7d29665 Update CHANGES.rst [ci skip]
    • d25036f Credits
    • 973a4c3 Release notes for 8.1.1
    • 521dab9 Use more specific regex chars to prevent ReDoS
    • 8b8076b Fix for CVE-2021-25291
    • e25be1e Fix negative size read in TiffDecode.c
    • f891baa Fix OOB read in SgiRleDecode.c
    • cbfdde7 Incorrect error code checking in TiffDecode.c
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 1
  • Bump ini from 1.3.5 to 1.3.8

    Bump ini from 1.3.5 to 1.3.8

    Bumps ini from 1.3.5 to 1.3.8.

    Commits
    • a2c5da8 1.3.8
    • af5c6bb Do not use Object.create(null)
    • 8b648a1 don't test where our devdeps don't even work
    • c74c8af 1.3.7
    • 024b8b5 update deps, add linting
    • 032fbaf Use Object.create(null) to avoid default object property hazards
    • 2da9039 1.3.6
    • cfea636 better git push script, before publish instead of after
    • 56d2805 do not allow invalid hazardous string as section name
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by isaacs, a new releaser for ini since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.19.2 to 0.21.1

    Bump axios from 0.19.2 to 0.21.1

    Bumps axios from 0.19.2 to 0.21.1.

    Release notes

    Sourced from axios's releases.

    v0.21.1

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    v0.21.0

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • What does Eposide id correspond to in a recommendation dataset

    What does Eposide id correspond to in a recommendation dataset

    I have a recommender dataset where I have userID, user_feature1,user_featureX...., ItemID,Item_feature1...Item_featureY.., reward(sales amount),datetime.. how should I restructure this data to be used/tested in Minerva? In the tutorial, https://minerva-ui.readthedocs.io/en/v0.40/getting_started.html, I am unable to correlate what will be the eposide ID in my dataset.

    opened by ayush488 0
  • ERROR: (sqlite3.OperationalError) unable to open database file.

    ERROR: (sqlite3.OperationalError) unable to open database file.

    Hi,

    I just installed the Minerva. However when I execute minerva run command I receive following error:

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/14/e3q8)

    Can you please tell me what is causing this error and how can I fix it? Thanks in advance.

    bug 
    opened by Zaharah 3
Releases(v0.40)
  • v0.40(May 29, 2021)

    Support d3rlpy v0.90!

    MINERVA now supports d3rlpy v0.90 and there are the highlights for MINERVA.

    • more precise CQL and BEAR implementations
    • predict_value is fixed for unscaled actions
    • CRR algorithm is available
    Source code(tar.gz)
    Source code(zip)
  • v0.32(Mar 6, 2021)

    a way to upload images is changed

    Previously, the image files are uploaded via selecting the directory that contains all the files. However this is an easy way, the browser limits the number of files due to the efficiency issue. So, from this version, you can upload zipped file that contains the image files. This is the more efficient and fast way. image

    Source code(tar.gz)
    Source code(zip)
  • v0.31(Mar 6, 2021)

  • v0.30(Mar 5, 2021)

    Support d3rlpy v0.70!

    MINERVA now supports d3rlpy v0.70, which has a lot of progress from the v0.41 that is the previous dependency.

    These are some highlights related to MINERVA.

    • the continuous action is automatically normalized based on the dataset statistics
    • discrete action option has been removed since the action-space is automatically detected
    • many other internal enhancements
    Source code(tar.gz)
    Source code(zip)
  • v0.20(Dec 21, 2020)

    Support d3rlpy v0.41!

    MINERVA now supports d3rlpy v0.41, which has a lot of progress from the v0.23 that is the previous dependency.

    These are the some highlights related to MINERVA.

    • extremely fast mini-batch creation
    • extremely fast frame stacking for image observation
    • extremely fast N-step TD calculation
    • new metrics
    • etc

    Algorithm Selection

    MINERVA now provides many many algorithms for both discrete and continuous control datasets. You can choose an algorithm at the project creation dialog.

    discrete algorithms

    • DQN
    • Double DQN
    • AWR
    • CQL
    • BCQ
    • SAC

    continuous algorithms

    • DDPG
    • TD3
    • SAC
    • BCQ
    • BEAR
    • CQL
    • AWR
    • AWAC
    • PLAS

    Of course, there is the Q function option to incorporate arbitrary algorithms with the powerful distributional Q functions.

    Q functions

    • mean
    • Quantile Regression
    • Implicit Quantile Network
    • Fully parameterized Quantile Function
    Source code(tar.gz)
    Source code(zip)
  • v0.12(Sep 9, 2020)

  • v0.11(Sep 9, 2020)

    Dataset Dashboard

    • upload datasets
      • discrete/continuous action-space
      • vector/image observations

    Project Dashboard

    • train with CQL with detailed configurations.

    Export Policy

    • Support TorchScript and ONNX.
    Source code(tar.gz)
    Source code(zip)
Owner
Takuma Seno
Machine learning engineer at Sony AI / Ph.D CS student at Keio University.
Takuma Seno
This folder contains the implementation of the multi-relational attribute propagation algorithm.

MrAP This folder contains the implementation of the multi-relational attribute propagation algorithm. It requires the package pytorch-scatter. Please

6 Dec 06, 2022
ProjectOxford-ClientSDK - This repo has moved :house: Visit our website for the latest SDKs & Samples

This project has moved 🏠 We heard your feedback! This repo has been deprecated and each project has moved to a new home in a repo scoped by API and p

Microsoft 970 Nov 28, 2022
ReferFormer - Official Implementation of ReferFormer

The official implementation of the paper: Language as Queries for Referring Vide

Jonas Wu 232 Dec 29, 2022
Fast and robust certifiable relative pose estimation

Fast and Robust Relative Pose Estimation for Calibrated Cameras This repository contains the code for the relative pose estimation between two central

42 Dec 06, 2022
PyTorch implementation for the ICLR 2020 paper "Understanding the Limitations of Variational Mutual Information Estimators"

Smoothed Mutual Information ``Lower Bound'' Estimator PyTorch implementation for the ICLR 2020 paper Understanding the Limitations of Variational Mutu

50 Nov 09, 2022
Fast and robust clustering of point clouds generated with a Velodyne sensor.

Depth Clustering This is a fast and robust algorithm to segment point clouds taken with Velodyne sensor into objects. It works with all available Velo

Photogrammetry & Robotics Bonn 957 Dec 21, 2022
LSTM and QRNN Language Model Toolkit for PyTorch

LSTM and QRNN Language Model Toolkit This repository contains the code used for two Salesforce Research papers: Regularizing and Optimizing LSTM Langu

Salesforce 1.9k Jan 08, 2023
The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp.

PISE The code for our CVPR paper PISE: Person Image Synthesis and Editing with Decoupled GAN, Project Page, supp. Requirement conda create -n pise pyt

jinszhang 110 Nov 21, 2022
PyQt6 configuration in yaml format providing the most simple script.

PyamlQt(ぴゃむるきゅーと) PyQt6 configuration in yaml format providing the most simple script. Requirements yaml PyQt6, ( PyQt5 ) Installation pip install Pya

Ar-Ray 7 Aug 15, 2022
Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation

Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation The code repository for "Audio-Visual Generalized Few-Shot Learning with

Kaiaicy 3 Jun 27, 2022
Official implementation for paper: Feature-Style Encoder for Style-Based GAN Inversion

Feature-Style Encoder for Style-Based GAN Inversion Official implementation for paper: Feature-Style Encoder for Style-Based GAN Inversion. Code will

InterDigital 63 Jan 03, 2023
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
Implementation of UNET architecture for Image Segmentation.

Semantic Segmentation using UNET This is the implementation of UNET on Carvana Image Masking Kaggle Challenge About the Dataset This dataset contains

Anushka agarwal 4 Dec 21, 2021
🛠 All-in-one web-based IDE specialized for machine learning and data science.

All-in-one web-based development environment for machine learning Getting Started • Features & Screenshots • Support • Report a Bug • FAQ • Known Issu

Machine Learning Tooling 2.9k Jan 09, 2023
PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"

FIERY This is the PyTorch implementation for inference and training of the future prediction bird's-eye view network as described in: FIERY: Future In

Wayve 406 Dec 24, 2022
pytorchのスライス代入操作をonnxに変換する際にScatterNDならないようにするサンプル

pytorch_remove_ScatterND pytorchのスライス代入操作をonnxに変換する際にScatterNDならないようにするサンプル。 スライスしたtensorにそのまま代入してしまうとScatterNDになるため、計算結果をcatで新しいtensorにする。 python ver

2 Dec 01, 2022
Efficient 6-DoF Grasp Generation in Cluttered Scenes

Contact-GraspNet Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, Dieter

NVIDIA Research Projects 148 Dec 28, 2022
Defending graph neural networks against adversarial attacks (NeurIPS 2020)

GNNGuard: Defending Graph Neural Networks against Adversarial Attacks Authors: Xiang Zhang ( Zitnik Lab @ Harvard 44 Dec 07, 2022