EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

Overview

Introduction EEGEyeNet

EEGEyeNet is a benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty.

Overview

The repository consists of general functionality to run the benchmark and custom implementation of different machine learning models. We offer to run standard ML models (e.g. kNN, SVR, etc.) on the benchmark. The implementation can be found in the StandardML_Models directory.

Additionally, we implemented a variety of deep learning models. These are implemented and can be run in both pytorch and tensorflow.

The benchmark consists of three tasks: LR (left-right), Direction (Angle, Amplitude) and Coordinates (x,y)

Installation (Environment)

There are many dependencies in this benchmark and we propose to use anaconda as package manager.

You can install a full environment to run all models (standard machine learning and deep learning models in both pytorch and tensorflow) from the eegeyenet_benchmark.yml file. To do so, run:

conda env create -f eegeyenet_benchmark.yml

Otherwise you can also only create a minimal environment that is able to run the models that you want to try (see following section).

General Requirements

Create a new conda environment:

conda create -n eegeyenet_benchmark python=3.8.5 

First install the general_requirements.txt

conda install --file general_requirements.txt 

Pytorch Requirements

If you want to run the pytorch DL models, first install pytorch in the recommended way. For Linux users with GPU support this is:

conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch 

For other installation types and cuda versions, visit pytorch.org.

Tensorflow Requirements

If you want to run the tensorflow DL models, run

conda install --file tensorflow_requirements.txt 

Standard ML Requirements

If you want to run the standard ML models, run

conda install --file standard_ml_requirements.txt 

This should be installed after installing pytorch to not risk any dependency issues that have to be resolved by conda.

Configuration

The model configuration takes place in hyperparameters.py. The training configuration is contained in config.py.

config.py

We start by explaining the settings that can be made for running the benchmark:

Choose the task to run in the benchmark, e.g.

config['task'] = 'LR_task'

For some tasks we offer data from multiple paradigms. Choose the dataset used for the task, e.g.

config['dataset'] = 'antisaccade'

Choose the preprocessing variant, e.g.

config['preprocessing'] = 'min'

Choose data preprocessed with Hilbert transformation. Set to True for the standard ML models:

config['feature_extraction'] = True

Include our standard ML models into the benchmark run:

config['include_ML_models'] = True 

Include our deep learning models into the benchmark run:

config['include_DL_models'] = True

Include your own models as specified in hyperparameters.py. For instructions on how to create your own custom models see further below.

config['include_your_models'] = True

Include dummy models for comparison into the benchmark run:

config['include_dummy_models'] = True

You can either choose to train models or use existing ones in /run/ and perform inference with them. Set

config['retrain'] = True 
config['save_models'] = True 

to train your specified models. Set both to False if you want to load existing models and perform inference. In this case specify the path to your existing model directory under

config['load_experiment_dir'] = path/to/your/model 

In the model configuration section you can specify which framework you want to use. You can run our deep learning models in both pytorch and tensorflow. Just specify it in config.py, make sure you set up the environment as explained above and everything specific to the framework will be handled in the background.

config.py also allows to configure hyperparameters such as the learning rate, and enable early stopping of models.

hyperparameters.py

Here we define our models. Standard ML models and deep learning models are configured in a dictionary which contains the object of the model and hyperparameters that are passed when the object is instantiated.

You can add your own models in the your_models dictionary. Specify the models for each task separately. Make sure to enable all the models that you want to run in config.py.

Running the benchmark

Create a /runs directory to save files while running models on the benchmark.

benchmark.py

In benchmark.py we load all models specified in hyperparameters.py. Each model is fitted and then evaluated with the scoring function corresponding to the task that is benchmarked.

main.py

To start the benchmark, run

python3 main.py

A directory of the current run is created, containing a training log, saving console output and model checkpoints of all runs.

Add Custom Models

To benchmark models we use a common interface we call trainer. A trainer is an object that implements the following methods:

fit() 
predict() 
save() 
load() 

Implementation of custom models

To implement your own custom model make sure that you create a class that implements the above methods. If you use library models, make sure to wrap them into a class that implements above interface used in our benchmark.

Adding custom models to our benchmark pipeline

In hyperparameters.py add your custom models into the your_models dictionary. You can add objects that implement the above interface. Make sure to enable your custom models in config.py.

Owner
Ard Kastrati
Ard Kastrati
A neuroanatomy-based augmented reality experience powered by computer vision. Features 3D visuals of the Atlas Brain Map slices.

Brain Augmented Reality (AR) A neuroanatomy-based augmented reality experience powered by computer vision that features 3D visuals of the Atlas Brain

Yasmeen Brain 10 Oct 06, 2022
Official Implementation for "StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery" (ICCV 2021 Oral)

StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery (ICCV 2021 Oral) Run this model on Replicate Optimization: Global directions: Mapper: Check ou

3.3k Jan 05, 2023
CoRe: Contrastive Recurrent State-Space Models

CoRe: Contrastive Recurrent State-Space Models This code implements the CoRe model and reproduces experimental results found in Robust Robotic Control

Apple 21 Aug 11, 2022
Research code for the paper "Variational Gibbs inference for statistical estimation from incomplete data".

Variational Gibbs inference (VGI) This repository contains the research code for Simkus, V., Rhodes, B., Gutmann, M. U., 2021. Variational Gibbs infer

Vaidotas Šimkus 1 Apr 08, 2022
Pytorch implementation of ProjectedGAN

ProjectedGAN-pytorch Pytorch implementation of ProjectedGAN (https://arxiv.org/abs/2111.01007) Note: this repository is still under developement. @InP

Dominic Rampas 17 Dec 14, 2022
Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

STARS Laboratory 8 Sep 14, 2022
Real-Time and Accurate Full-Body Multi-Person Pose Estimation&Tracking System

News! Aug 2020: v0.4.0 version of AlphaPose is released! Stronger tracking! Include whole body(face,hand,foot) keypoints! Colab now available. Dec 201

Machine Vision and Intelligence Group @ SJTU 6.7k Dec 28, 2022
On the Limits of Pseudo Ground Truth in Visual Camera Re-Localization

On the Limits of Pseudo Ground Truth in Visual Camera Re-Localization This repository contains the evaluation code and alternative pseudo ground truth

Torsten Sattler 36 Dec 22, 2022
Implementation for NeurIPS 2021 Submission: SparseFed

READ THIS FIRST This repo is an anonymized version of an existing repository of GitHub, for the AIStats 2021 submission: SparseFed: Mitigating Model P

2 Jun 15, 2022
Split your patch similarly to `git add -p` but supporting multiple buckets

split-patch.py This is git add -p on steroids for patches. Given a my.patch you can run ./split-patch.py my.patch You can choose in which bucket to p

102 Oct 06, 2022
Spatio-Temporal Entropy Model (STEM) for end-to-end leaned video compression.

Spatio-Temporal Entropy Model A Pytorch Reproduction of Spatio-Temporal Entropy Model (STEM) for end-to-end leaned video compression. More details can

16 Nov 28, 2022
⚾🤖⚾ Automatic baseball pitching overlay in realtime

⚾ Automatically overlaying pitch motion and trajectory with machine learning! This project takes your baseball pitching clips and automatically genera

Tony Chou 240 Dec 05, 2022
Plenoxels: Radiance Fields without Neural Networks

Plenoxels: Radiance Fields without Neural Networks Alex Yu*, Sara Fridovich-Keil*, Matthew Tancik, Qinhong Chen, Benjamin Recht, Angjoo Kanazawa UC Be

Sara Fridovich-Keil 81 Dec 25, 2022
CTC segmentation python package

CTC segmentation CTC segmentation can be used to find utterances alignments within large audio files. This repository contains the ctc-segmentation py

Ludwig Kürzinger 217 Jan 04, 2023
(NeurIPS 2020) Wasserstein Distances for Stereo Disparity Estimation

Wasserstein Distances for Stereo Disparity Estimation Accepted in NeurIPS 2020 as Spotlight. [Project Page] Wasserstein Distances for Stereo Disparity

Divyansh Garg 92 Dec 12, 2022
Official implementation of ACMMM'20 paper 'Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework'

Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework Official code for paper, Self-supervised Video Representation Le

Li Tao 103 Dec 21, 2022
Sarus implementation of classical ML models. The models are implemented using the Keras API of tensorflow 2. Vizualization are implemented and can be seen in tensorboard.

Sarus published models Sarus implementation of classical ML models. The models are implemented using the Keras API of tensorflow 2. Vizualization are

Sarus Technologies 39 Aug 19, 2022
Faster Convex Lipschitz Regression

Faster Convex Lipschitz Regression This reepository provides a python implementation of our Faster Convex Lipschitz Regression algorithm with GPU and

Ali Siahkamari 0 Nov 19, 2021
Download and preprocess popular sequential recommendation datasets

Sequential Recommendation Datasets This repository collects some commonly used sequential recommendation datasets in recent research papers and provid

125 Dec 06, 2022
This is the code used in the paper "Entity Embeddings of Categorical Variables".

This is the code used in the paper "Entity Embeddings of Categorical Variables". If you want to get the original version of the code used for the Kagg

Cheng Guo 845 Nov 29, 2022