Testbed of AI Systems Quality Management

Related tags

Deep Learningpythondockerflaskairflowvuejspostgresqljupyter-notebookqualitiesmanaging-ai
Overview

qunomon

Description

A testbed for testing and managing AI system qualities.

Demo

Sorry. Not deployment public server at alpha version.

Requirement

Installation prerequisites

Support os is Windows10 Pro and macOS.

  • Windows10 Pro 1909 later
  • macOS v10.15 later

Installation

Usage

1.launch

Execute the following command as root of this repository.

docker-compose up

2.access web browser

http://127.0.0.1:8888/

Development for windows

Installation

1.PackageManager

  • Launch powershell with administrator permission.

  • powershell

    Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))

2.Python

  • powershell 
    cinst python --version=3.6.8 -y

Setup python virtual environment for Backend

1.go to the source you checked out and create a virtual environment

  • launch command prompt
cd {checkout_dir}\src\backend
python -m venv venv

2.virtual environment activate

.\venv\Scripts\activate

3.install python package

pip install -r requirements_dev.txt

Setup python virtual environment for IP

1.go to the source you checked out and create a virtual environment

  • launch command prompt
cd {checkout_dir}\src\integration-provider
python -m venv venv

2.virtual environment activate

.\venv\Scripts\activate

3.install python package

pip install -r constraints.txt

launch by without container

1.execute bat file

start_up.bat

2.checking web browser

http://127.0.0.1:8080/

3.checking Backend

  • powershell 
    curl http://127.0.0.1:5000/qai-testbed/api/0.0.1/health-check

4.checking IP

  • powershell 
    curl http://127.0.0.1:6000/qai-ip/api/0.0.1/health-check

Contribution

Bug reports and pull requests are welcome on GitHub at aistairc/qunomon.

Disclaimer

qunomon is an OSS and alpha version. so qunomon may cause damage to your system and data. You agree to use it at your own risk.

License

Apache License Version 2.0

Author

AIST

You might also like...
Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment (ICCV2021)
Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment (ICCV2021)

Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment This is a pytorch project for the paper Seeing Dynamic Scene i

DV Lab 21 Nov 28, 2022
Official PyTorch implementation of the paper
Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

RecycleD Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN

Yunan Zhu 23 Nov 5, 2022
[ICCV 2021] Group-aware Contrastive Regression for Action Quality Assessment
[ICCV 2021] Group-aware Contrastive Regression for Action Quality Assessment

CoRe Created by Xumin Yu*, Yongming Rao*, Wenliang Zhao, Jiwen Lu, Jie Zhou This is the PyTorch implementation for ICCV paper Group-aware Contrastive

Xumin Yu 31 Dec 24, 2022
NUANCED is a user-centric conversational recommendation dataset that contains 5.1k annotated dialogues and 26k high-quality user turns.
NUANCED is a user-centric conversational recommendation dataset that contains 5.1k annotated dialogues and 26k high-quality user turns.

NUANCED: Natural Utterance Annotation for Nuanced Conversation with Estimated Distributions Overview NUANCED is a user-centric conversational recommen

Facebook Research 18 Dec 28, 2021
Cross Quality LFW: A database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments

Cross-Quality Labeled Faces in the Wild (XQLFW) Here, we release the database, evaluation protocol and code for the following paper: Cross Quality LFW

Martin Knoche 10 Dec 12, 2022
A data annotation pipeline to generate high-quality, large-scale speech datasets with machine pre-labeling and fully manual auditing.
A data annotation pipeline to generate high-quality, large-scale speech datasets with machine pre-labeling and fully manual auditing.

About This repository provides data and code for the paper: Scalable Data Annotation Pipeline for High-Quality Large Speech Datasets Development (subm

Appen Repos 86 Dec 7, 2022
PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech
PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech

PortaSpeech - PyTorch Implementation PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech. Model Size Module Nor

Keon Lee 279 Jan 4, 2023
Code for the paper SphereRPN: Learning Spheres for High-Quality Region Proposals on 3D Point Clouds Object Detection, ICIP 2021.

SphereRPN Code for the paper SphereRPN: Learning Spheres for High-Quality Region Proposals on 3D Point Clouds Object Detection, ICIP 2021. Authors: Th

Thang Vu 15 Dec 2, 2022
Implementation of temporal pooling methods studied in [ICIP'20] A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment

Implementation of temporal pooling methods studied in [ICIP'20] A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment

Zhengzhong Tu 5 Sep 16, 2022
Releases(0.1.15)

  • 0.1.15(Jun 25, 2021)

  • 0.1.14(Jun 8, 2021)

    Added

    #1462 SHAP AITの実装とテスト alyz_regression_shap_0.1 #1485 SHAP AIT plots_scatterの出力figにタイトル（カラムを対象）追加

    Fixed

    #1492 Dependabot alerts対応（urllib3） #1476 Dependabot alerts対応（TensorFlow2.4系から変更） #1465 AITパラメータ見直し（eval_adversarial_example_acc_test_tf2.3_0.1）

    Source code(tar.gz)
    Source code(zip)

  • 0.1.13(May 26, 2021)

    Added

    #1434 クライテリア範囲外でTDを作成できないようにする(バックエンド) #1435 クライテリア範囲外でTDを作成できないようにする(フロントエンド) #1436 パラメータ範囲外でTDを作成できないようにする(フロントエンド) #1437 パラメータ範囲外でTDを作成できないようにする(バックエンド) #1438 インベントリチェック　警告ポップアップを表示する（フロントエンド） #1440 インベントリチェック　ファイルフォーマットチェック(一般)

    Fixed

    #1423 AITパラメータ見直し（eval_dnn_coverage_tf1.13_0.1） #1425 AITパラメータ見直し（eval_mnist_acc_tf2.3_0.1） #1454 DependencyAlert解消(5/12)

    Source code(tar.gz)
    Source code(zip)

  • 0.1.12(May 12, 2021)

    Added

    #1416 インベントリチェック　ファイル存在チェック #1421 インベントリチェック　TD実行時ハッシュチェック

    Fixed

    #1362 #1213の変更部分をテストコードに反映させる #1403 AIT発生エラー見直し AIT-SDK入れ替え #1432 GET TestRunnerでエンコードエラーログが出力される #1442 docker起動でインベントリ登録ができない #1447 DependencyAlert解消(5/10) #1452 pipインストールモジュールのバージョンを固定化する

    Source code(tar.gz)
    Source code(zip)

  • 0.1.11(Apr 28, 2021)

    Added

    #1370 AITの更新 (AITのパラメータ上限下限を表示する) #1374 ait-installerの更新 (AITのパラメータ上限下限を表示する)

    Fixed

    #1402 AIT発生エラー見直し AIT-SDK修正 #1404 AIT発生エラー見直し IP修正 #1405 AIT発生エラー見直し バックエンド修正 #1406 AIT発生エラー見直し フロントエンド修正 #1416 AIT発生エラー見直し AIT-SDK修正（出力先フォルダがない場合に対応）

    Source code(tar.gz)
    Source code(zip)

  • 0.1.10(Apr 14, 2021)

    Added

    #1349 [TF3]MLComponent一覧画面でMLComponentを削除できるようにしたい

    Fixed

    #1385 DependencyAlert解消(3/26) #1391 DependencyAlert解消(4/2) #1361 #1335 により変更された部分をWEB API仕様書に反映をさせる

    Source code(tar.gz)
    Source code(zip)

  • 0.1.9(Mar 26, 2021)

    Added

    #1335 AITのパラメータ上限下限を表示する #144 TestDescription一覧画面の日付指定をカレンダーを用いて行う機能の実装

    Fixed

    なし

    Source code(tar.gz)
    Source code(zip)

  • 0.1.8(Mar 12, 2021)

    Added

    #1212 TD詳細画面-グラフを複数選択して追加したい #1340 TD詳細画面でairflowのログダウンロードURLリンクを表示する #1342 グラフ複数選択時に未登録のものだけを登録したい #1347 [TF3]TD一覧画面でTDを削除できるようにしたい

    Fixed

    #1331 何も選択していない状態で「add to Report」ボタンを押下できてしまう #1337 活性化判定をcheckAddBTNActiveメソッドで対応させるよう処理を統一 #1345 [TD詳細画面]追加グラフの数チェック不整合 #1348 [TD編集画面1]TDの再編集時にTD名のテキストボックスが入力1文字ごとにフォーカスが外れる #1336 docker-compose実行時に、ait-installerが実行されてない

    Source code(tar.gz)
    Source code(zip)

  • 0.1.7(Feb 26, 2021)

  • 0.1.6(Feb 12, 2021)

    Added

    #1200 AIF360の指標を取り込んだAITを作成する #1211 TD詳細画面-どのグラフを選択中か分かるようにしたい #1248 ait.manifest.jsonのreport.measuresにminとmaxを書く

    Fixed

    #1300 jupyter新バージョン3.X以後、AITのset_ait_descriptionにUnicodeEncodeError (漢字、など) #1305 TDでのレポート使用グラフを一つ削除すると、ソートがリセットされる #1312 Dependency alert解消(2/4)

    Source code(tar.gz)
    Source code(zip)

  • 0.1.5(Jan 29, 2021)

    Added

    #1199 TDの品質指標に何を入れれば良いか分かりにくい問題を解消する #1213 [サマリ]manifestのresources,downloadsからpathを削除する

    Fixed

    #1262 eval_bdd100k_aicc_tf2.3のリソース「all_label_accuracy_csv」がタイプ「text」になっている #1272 ローカルにAITイメージがない状態で実行するとairflowでエラーになる #1277 Dependabot alerts解消(2021/1/15)

    Source code(tar.gz)
    Source code(zip)

  • 0.1.4(Jan 15, 2021)

    Added

    #1213 manifestのresources,downloadsからpathを削除する

    Fixed

    #1208 レポートのレーダーチャートが、品質特性2以下だと数量が判別できない #1254 レポートのレーダーチャートの表示範囲が５で固定 #1260 Dependabot alerts解消(2021/1/8)

    Source code(tar.gz)
    Source code(zip)

  • 0.1.3(Dec 25, 2020)

    Added

    #1166 qlib新規作成

    Fixed

    #1183 TestDescriptionの中で大量の画像を扱うと画面が応答しない #1198 フロントエンド誤字修正 #1203 AITでresoucesに大量のデータをセットすると、TestDescription詳細画面やレポート出力が応答しない #1242 measures無しのAITを登録するとQualityDimensionが反映されない

    Source code(tar.gz)
    Source code(zip)

  • 0.1.2(Dec 10, 2020)

    Added

    #1171 インベントリの選択方法を改善する #1173 レポートのサマリでTD0件の品質特性は出力対象にしないようにする

    Fixed

    #1187 レポート出力時に2.1のレーダーチャートの項目名が長すぎると途中で切れる #1184 airflowのdocker buildが失敗する

    Source code(tar.gz)
    Source code(zip)

  • 0.1.1(Nov 27, 2020)

    Added

    #1071 確認ダイアログの多言語化対応 #1126 作成したAITをtestbedにdeployするツールが必要

    Fixed

    #1099 ブラウザバック、リロードでエラーが発生する画面がある #1123 内部品質名称を英語に変更する #1115 2つ以上あるmeasureのうち、一つだけチェックをいれてTDを作成するとエラーが発生する #1125 READMEの記述を修正（qai-testbed → qunomon） #1163 dag配下のフォルダを削除する #1156 docker-airflowのDockerfileの修正 #1157 Github security alert への対応

    Source code(tar.gz)
    Source code(zip)

  • 0.1.0(Nov 13, 2020)

Owner
AIST AIRC
AIST AIRC
GitHub Repository https://aistairc.github.io/qunomon/
CONditionals for Ordinal Regression and classification in tensorflow

Condor Ordinal regression in Tensorflow Keras Tensorflow Keras implementation of CONDOR Ordinal Regression (aka ordinal classification) by Garrett Jen

9 Jul 31, 2022
working repo for my xumx-sliCQ submissions to the ISMIR 2021 MDX

Music Demixing Challenge - xumx-sliCQ This repository is the GitHub mirror of my working submission repository for the AICrowd ISMIR 2021 Music Demixi

4 Aug 25, 2021
A learning-based data collection tool for human segmentation

FullBodyFilter A Learning-Based Data Collection Tool For Human Segmentation Contents Documentation Source Code and Scripts Overview of Project Usage O

Robert Jiang 4 Jun 24, 2022
This is an implementation of PIFuhd based on Pytorch

Open-PIFuhd This is a unofficial implementation of PIFuhd PIFuHD: Multi-Level Pixel-Aligned Implicit Function forHigh-Resolution 3D Human Digitization

Lingteng Qiu 235 Dec 19, 2022
An onlinel learning to rank python codebase.

OLTR Online learning to rank python codebase. The code related to Pairwise Differentiable Gradient Descent (ranker/PDGDLinearRanker.py) is copied from

ielab 5 Jul 18, 2022
Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression

Alpha-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression YOLOv5 with alpha-IoU losses implemented in PyTorch. Example r

Jacobi(Jiabo He) 147 Dec 05, 2022
Investigating Attention Mechanism in 3D Point Cloud Object Detection (arXiv 2021)

Investigating Attention Mechanism in 3D Point Cloud Object Detection (arXiv 2021) This repository is for the following paper: "Investigating Attention

52 Nov 19, 2022
Implementation of CVPR'21: RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

RfD-Net [Project Page] [Paper] [Video] RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction Yinyu Nie, Ji Hou, Xiaoguang Han, Matthi

Yinyu Nie 162 Jan 06, 2023
A Keras implementation of YOLOv3 (Tensorflow backend)

keras-yolo3 Introduction A Keras implementation of YOLOv3 (Tensorflow backend) inspired by allanzelener/YAD2K. Quick Start Download YOLOv3 weights fro

7.1k Jan 03, 2023
Code for paper "Context-self contrastive pretraining for crop type semantic segmentation"

Code for paper "Context-self contrastive pretraining for crop type semantic segmentation" Setting up a python environment Follow the instruction in ht

Michael Tarasiou 11 Oct 09, 2022
ADOP: Approximate Differentiable One-Pixel Point Rendering

ADOP: Approximate Differentiable One-Pixel Point Rendering Abstract: We present a novel point-based, differentiable neural rendering pipeline for scen

Darius Rückert 1.9k Jan 06, 2023
Using contrastive learning and OpenAI's CLIP to find good embeddings for images with lossy transformations

Creating Robust Representations from Pre-Trained Image Encoders using Contrastive Learning Sriram Ravula, Georgios Smyrnis This is the code for our pr

Sriram Ravula 26 Dec 10, 2022
Code for NeurIPS 2021 paper "Curriculum Offline Imitation Learning"

README The code is based on the ILswiss. To run the code, use python run_experiment.py --nosrun -e your YAML file -g gpu id Generally, run_experim

ApexRL 12 Mar 19, 2022
Point Cloud Registration Network

PCRNet: Point Cloud Registration Network using PointNet Encoding Source Code Author: Vinit Sarode and Xueqian Li Paper | Website | Video | Pytorch Imp

ViNiT SaRoDe 59 Nov 19, 2022
Sequence-to-Sequence learning using PyTorch

Seq2Seq in PyTorch This is a complete suite for training sequence-to-sequence models in PyTorch. It consists of several models and code to both train

Elad Hoffer 514 Nov 17, 2022
ICSS - Interactive Continual Semantic Segmentation

Presentation This repository contains the code of our paper: Weakly-supervised c

Alteia 9 Jul 23, 2022
Api's bulid in Flask perfom to manage Todo Task.

Citymall-task Api's bulid in Flask perfom to manage Todo Task. Installation Requrements : Python: 3.10.0 MongoDB create .env file with variables DB_UR

Aisha Tayyaba 1 Dec 17, 2021
Bridging the Gap between Label- and Reference based Synthesis(ICCV 2021)

Bridging the Gap between Label- and Reference based Synthesis(ICCV 2021) Tensorflow implementation of Bridging the Gap between Label- and Reference-ba

huangqiusheng 8 Jul 13, 2022
Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Memory Efficient Attention Pytorch Implementation of a memory efficient multi-head attention as proposed in the paper, Self-attention Does Not Need O(

Phil Wang 180 Jan 05, 2023
Keras implementation of AdaBound

AdaBound for Keras Keras port of AdaBound Optimizer for PyTorch, from the paper Adaptive Gradient Methods with Dynamic Bound of Learning Rate. Usage A

Somshubra Majumdar 132 Sep 23, 2022