BErt-like Neurophysiological Data Representation

Related tags

Data AnalysisBENDR
Overview

BENDR

BErt-like Neurophysiological Data Representation

A picture of Bender from Futurama

This repository contains the source code for reproducing, or extending the BERT-like self-supervision pre-training for EEG data from the article:

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

To run these scripts, you will need to use the DN3 project. We will try to keep this updated so that it works with the latest DN3 release. If you are just looking for the BENDR model, and don't need to reproduce the article results per se, BENDR will be (or maybe already is if I forgot to update it here) integrated into DN3, in which case I would start there.

Currently, we recommend version 0.2. Feel free to open an issue if you are having any trouble.

More extensive instructions are upcoming, but in essence you will need to either:

a)  Download the TUEG dataset and pre-train new encoder and contextualizer weights, _or_
b)  Use the [pre-trained model weights](https://github.com/SPOClab-ca/BENDR/releases/tag/v0.1-alpha)

Once you have a pre-trained model:

1) Add the paths of the pre-trained weights to configs/downstream.yml
2) Edit paths to local copies of your datasets in configs/downstream_datasets.yml
3) Run downstream.sh

Comments
  • about the loss function

    about the loss function

    Very appreciate for your contribution.i am really interested in the self training in EEG. The only question is about calculating loss function. In your paper, The calculation of the denominator uses cosine similarity between the output of the transformer and the 20 distractors and the input of the transformer. However, in the code, the calculation of the denominator uses cosine similarity between the input of the transformer and the 20 distractors, and the output of the transformer. In other word, the output and the input switch positions. Are both the calculation approaches the same? Or why did you change the calculation approache in the code? Thanks!

    opened by stickOverCarrot 2
  • About deploy downstream.yml and downstream_datasets.yml

    About deploy downstream.yml and downstream_datasets.yml

    Tranks for supplying your code. But when I follow your markdown, I meet some problems image

    This is my project files image

    This is my downstream.yml image

    This is my downstream_datasets.yml image

    opened by YoloEliwa 1
  • Pre-trained weights?

    Pre-trained weights?

    Not an issue per se, but you state the pre-trained weights for your paper are available in this repo, yet I have had a good look around and I haven't found them, nor a means of downloading them. Please can you let me know where I could find them? I'm really keen to try out this exciting architecture you've put together!

    opened by SgtWhiskeyjack 1
  • result_tracking module

    result_tracking module

    There's a reference that's in the module import: downstream.py from result_tracking import ThinkerwiseResultTracker that looks like some type of tracking code for experiments?

    opened by bencten 1
  • dropout should change

    dropout should change

    Iteration: 4%|▍ | 13/330 [00:36<16:00, 3.03s/batches, bac=0.5, Accuracy=0.51, loss=0.695, lr=1.47e-6]D:\Anaconda\envs\LGG\lib\site-packages\torch\nn\functional.py:1338: UserWarning: dropout2d: Received a 3D input to dropout2d and assuming that channel-wise 1D dropout behavior is desired - input is interpreted as shape (N, C, L), where C is the channel dim. This behavior will change in a future release to interpret the input as one without a batch dimension, i.e. shape (C, H, W). To maintain the 1D channel-wise dropout behavior, please switch to using dropout1d instead. warnings.warn("dropout2d: Received a 3D input to dropout2d and assuming that channel-wise "

    opened by zy2021314 0
  • A more detailed explanation

    A more detailed explanation

    We need to use your code for research, may I ask when you can provide detailed explanation, because we have some difficulties in understanding the code without detailed explanation.

    opened by EchizenMike 0
  • preload in downstream.yml

    preload in downstream.yml

    In the "downstream.yml" file, what is the function of the "preload"? What's mean if I specify "preload: True" or "preload: False"?

    Thank you in advance

    opened by frannfuri 0
Releases(v0.1-alpha)
Owner
GitHub Repository
Python Practicum - prepare for your Data Science interview or get a refresher.

Python-Practicum Python Practicum - prepare for your Data Science interview or get a refresher. Data Data visualization using data on births from the

Jovan Trajceski 1 Jul 27, 2021
The Spark Challenge Student Check-In/Out Tracking Script

The Spark Challenge Student Check-In/Out Tracking Script This Python Script uses the Student ID Database to match the entries with the ID Card Swipe a

1 Dec 09, 2021
Automated Exploration Data Analysis on a financial dataset

Automated EDA on financial dataset Just a simple way to get automated Exploration Data Analysis from financial dataset (OHLCV) using Streamlit and ta.

Darío López Padial 28 Nov 27, 2022
Python package for processing UC module spectral data.

UC Module Python Package How To Install clone repo. cd UC-module pip install . How to Use uc.module.UC(measurment=str, dark=str, reference=str, heade

Nicolai Haaber Junge 1 Oct 20, 2021
Python-based Space Physics Environment Data Analysis Software

pySPEDAS pySPEDAS is an implementation of the SPEDAS framework for Python. The Space Physics Environment Data Analysis Software (SPEDAS) framework is

SPEDAS 98 Dec 22, 2022
ETL flow framework based on Yaml configs in Python

ETL framework based on Yaml configs in Python A light framework for creating data streams. Setting up streams through configuration in the Yaml file.

Павел Максимов 18 Jul 06, 2022
Office365 (Microsoft365) audit log analysis tool

Office365 (Microsoft365) audit log analysis tool The header describes it all WHY?? The first line of code was written long time before other colleague

Anatoly 1 Jul 27, 2022
CubingB is a timer/analyzer for speedsolving Rubik's cubes, with smart cube support

CubingB is a timer/analyzer for speedsolving Rubik's cubes (and related puzzles). It focuses on supporting "smart cubes" (i.e. bluetooth cubes) for recording the exact moves of a solve in real time.

Zach Wegner 5 Sep 18, 2022
MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.

MetPy MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data. MetPy follows semantic versioni

Unidata 971 Dec 25, 2022
Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine

Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine Intro This repo contains the python/stan version of the Statistical Rethinking

Andrés Suárez 3 Nov 08, 2022
Finding project directories in Python (data science) projects, just like there R rprojroot and here packages

Find relative paths from a project root directory Finding project directories in Python (data science) projects, just like there R here and rprojroot

Daniel Chen 102 Nov 16, 2022
This is a repo documenting the best practices in PySpark.

Spark-Syntax This is a public repo documenting all of the "best practices" of writing PySpark code from what I have learnt from working with PySpark f

Eric Xiao 447 Dec 25, 2022
pandas: powerful Python data analysis toolkit

pandas is a Python package that provides fast, flexible, and expressive data structures designed to make working with "relational" or "labeled" data both easy and intuitive.

pandas 36.4k Jan 03, 2023
Data imputations library to preprocess datasets with missing data

Impyute is a library of missing data imputation algorithms. This library was designed to be super lightweight, here's a sneak peak at what impyute can do.

Elton Law 329 Dec 05, 2022
Orchest is a browser based IDE for Data Science.

Orchest is a browser based IDE for Data Science. It integrates your favorite Data Science tools out of the box, so you don’t have to. The application is easy to use and can run on your laptop as well

Orchest 3.6k Jan 09, 2023
InDels analysis of CRISPR lines by NGS amplicon sequencing technology for a multicopy gene family.

CRISPRanalysis InDels analysis of CRISPR lines by NGS amplicon sequencing technology for a multicopy gene family. In this work, we present a workflow

2 Jan 31, 2022
CS50 pset9: Using flask API to create a web application to exchange stocks' shares.

C$50 Finance In this guide we want to implement a website via which users can “register”, “login” “buy” and “sell” stocks, like below: Background If y

1 Jan 24, 2022
Universal data analysis tools for atmospheric sciences

U_analysis Universal data analysis tools for atmospheric sciences Script written in python 3. This file defines multiple functions that can be used fo

Luis Ackermann 1 Oct 10, 2021
Fast, flexible and easy to use probabilistic modelling in Python.

Please consider citing the JMLR-MLOSS Manuscript if you've used pomegranate in your academic work! pomegranate is a package for building probabilistic

Jacob Schreiber 3k Jan 02, 2023
This program analyzes a DNA sequence and outputs snippets of DNA that are likely to be protein-coding genes.

This program analyzes a DNA sequence and outputs snippets of DNA that are likely to be protein-coding genes.

1 Dec 28, 2021