Long Expressive Memory (LEM)

Last update: Dec 17, 2022

Overview

Long Expressive Memory for Sequence Modeling

This repository contains the implementation to reproduce the numerical experiments of the paper Long Expressive Memory for Sequence Modeling

Requirements

To setup the environment, install the requirements using python 3.7.10 with

pip install -r requirements.txt

Experiments

This repository contains the codes to reproduce the results of the following experiments for the proposed LEM:

Adding Task
EigenWorms
FitzHugh-Nagumo
Google12 (V2)
Heart rate prediction
noisy CIFAR-10
Permuted Sequential MNIST
PennTree Bank (word- and char-level)
Sequential MNIST

All datasets get either synthetically generated or downloaded automatically, providing a full data pipline for each experiment.
Note that the PTB experiments are based on the open-source tensorflow project LAnguage Modelling Benchmarks (lamb) for tuning and testing Tensorflow language models. In order to reproduce these experiments, please follow the steps explained in the README of the PTB directory.

Results

The results of LEM for each of the experiments are:

Experiment	Result
EigenWorms	92.3% mean test accuracy
FitzHugh-Nagumo	0.002 test L^2 loss
Google12 (V2)	95.7% test accuracy
Heart rate prediction	0.85 test L^2 loss
noisy CIFAR-10	60.5% test accuracy
Permuted Sequential MNIST	96.6% test accuracy
PennTree Bank char-level (single layer)	1.25 test bits-per-character
PennTree Bank word-level (single layer)	72.8 test perplexity
Sequential MNIST	99.5% test accuracy

Long Expressive Memory (LEM)

Related tags

Overview

Long Expressive Memory for Sequence Modeling

Requirements

Experiments

Results

Owner

Konstantin Rusch

ContourletNet: A Generalized Rain Removal Architecture Using Multi-Direction Hierarchical Representation

Meta Learning for Semi-Supervised Few-Shot Classification

The modify PyTorch version of Siam-trackers which are speed-up by TensorRT.

Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Calculates JMA (Japan Meteorological Agency) seismic intensity (shindo) scale from acceleration data recorded in NumPy array

An Exact Solver for Semi-supervised Minimum Sum-of-Squares Clustering

This is a project based on retinaface face detection, including ghostnet and mobilenetv3

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Shared Attention for Multi-label Zero-shot Learning

The description of FMFCC-A (audio track of FMFCC) dataset and Challenge resluts.

A collection of Reinforcement Learning algorithms from Sutton and Barto's book and other research papers implemented in Python.

Autolfads-tf2 - A TensorFlow 2.0 implementation of Latent Factor Analysis via Dynamical Systems (LFADS) and AutoLFADS

Its a Plant Leaf Disease Detection System based on Machine Learning.

Keras Image Embeddings using Contrastive Loss

Sparse R-CNN: End-to-End Object Detection with Learnable Proposals, CVPR2021

A simple configurable bot for sending arXiv article alert by mail

A framework that allows people to write their own Rocket League bots.

Exploring the Dual-task Correlation for Pose Guided Person Image Generation

Simple-System-Convert--C--F - Simple System Convert With Python

Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.