Efficient Two-Step Networks for Temporal Action Segmentation (Neurocomputing 2021)

Related tags

Deep LearningETSN
Overview

Efficient Two-Step Networks for Temporal Action Segmentation

This repository provides a PyTorch implementation of the paper Efficient Two-Step Networks for Temporal Action Segmentation.

Requirements

* Python 3.8.5
* pyTorch 1.8.1

You can download packages using requirements.txt.
pip install -r requirements.txt

Datasets

  • Download the data provided by MS-TCN, which contains the I3D features (w/o fine-tune) and the ground truth labels for 3 datasets. (~30GB)
  • Extract it so that you have the data folder in the same directory as train.py.

directory structure

├── config
│   ├── 50salads
│   ├── breakfast
│   └── gtea
├── csv
│   ├── 50salads
│   ├── breakfast
│   └── gtea
├─ dataset ─── 50salads/...
│           ├─ breakfast/...
│           └─ gtea ─── features/
│                    ├─ groundTruth/
│                    ├─ splits/
│                    └─ mapping.txt
├── libs
├── result
├── utils 
├── requirements.txt
├── train.py
├── eval.py
└── README.md

Training and Testing of ETSN

Setting

First, convert ground truth files into numpy array.

python utils/generate_gt_array.py ./dataset

Then, please run the below script to generate csv files for data laoder'.

python utils/builda_dataset.py ./dataset

Training

You can train a model by changing the settings of the configuration file.

python train.py ./config/xxx/xxx/config.yaml

Evaluation

You can evaluate the performance of result after running.

python eval.py ./result/xxx/xxx/config.yaml test

We also provide trained ETSN model in Google Drive. Extract it so that you have the result folder in the same directory as train.py.

average cross validation results

python utils/average_cv_results.py [result_dir]

Citation

If you find our code useful, please cite our paper.

@article{LI2021373,
author = {Yunheng Li and Zhuben Dong and Kaiyuan Liu and Lin Feng and Lianyu Hu and Jie Zhu and Li Xu and Yuhan wang and Shenglan Liu},
journal = {Neurocomputing},
title = {Efficient Two-Step Networks for Temporal Action Segmentation},
year = {2021},
volume = {454},
pages = {373-381},
issn = {0925-2312},
doi = {https://doi.org/10.1016/j.neucom.2021.04.121},
url = {https://www.sciencedirect.com/science/article/pii/S0925231221006998},

}

Contact

For any question, please raise an issue or contact.

Acknowledgement

We appreciate MS-TCN for extracted I3D feature, backbone network and evaluation code.

Appreciating Yuchi Ishikawa shares the re-implementation of MS-TCN with pytorch.

ViSD4SA, a Vietnamese Span Detection for Aspect-based sentiment analysis dataset

UIT-ViSD4SA PACLIC 35 General Introduction This repository contains the data of the paper: Span Detection for Vietnamese Aspect-Based Sentiment Analys

Nguyễn Thị Thanh Kim 5 Nov 13, 2022
PyTorch implementation for 3D human pose estimation

Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach This repository is the PyTorch implementation for the network presented in:

Xingyi Zhou 579 Dec 22, 2022
Gender Classification Machine Learning Model using Sk-learn in Python with 97%+ accuracy and deployment

Gender-classification This is a ML model to classify Male and Females using some physical characterstics Data. Python Libraries like Pandas,Numpy and

Aryan raj 11 Oct 16, 2022
Source code for our paper "Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures"

Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures Code for the Multiplex Molecular Graph Neural Network (M

shzhang 59 Dec 10, 2022
Simple transformer model for CIFAR10

CIFAR-Transformer Simple transformer model for CIFAR10. Reference: https://www.tensorflow.org/text/tutorials/transformer https://github.com/huggingfac

9 Nov 07, 2022
Segmentation vgg16 fcn - cityscapes

VGGSegmentation Segmentation vgg16 fcn - cityscapes Priprema skupa skripta prepare_dataset_downsampled.py Iz slika cityscapesa izrezuje haubu automobi

6 Oct 24, 2020
source code and pre-trained/fine-tuned checkpoint for NAACL 2021 paper LightningDOT

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval This repository contains source code and pre-trained/fine-tun

Siqi 65 Dec 26, 2022
Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research

Welcome to AirSim AirSim is a simulator for drones, cars and more, built on Unreal Engine (we now also have an experimental Unity release). It is open

Microsoft 13.8k Jan 05, 2023
An optimization and data collection toolbox for convenient and fast prototyping of computationally expensive models.

An optimization and data collection toolbox for convenient and fast prototyping of computationally expensive models. Hyperactive: is very easy to lear

Simon Blanke 422 Jan 04, 2023
A Keras implementation of CapsNet in the paper: Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. Dynamic Routing Between Capsules

NOTE This implementation is fork of https://github.com/XifengGuo/CapsNet-Keras , applied to IMDB texts reviews dataset. CapsNet-Keras A Keras implemen

Lauro Moraes 5 Oct 23, 2022
The Simplest DCGAN Implementation

DCGAN in TensorLayer This is the TensorLayer implementation of Deep Convolutional Generative Adversarial Networks. Looking for Text to Image Synthesis

TensorLayer Community 310 Dec 13, 2022
SmoothGrad implementation in PyTorch

SmoothGrad implementation in PyTorch PyTorch implementation of SmoothGrad: removing noise by adding noise. Vanilla Gradients SmoothGrad Guided backpro

SSKH 143 Jan 05, 2023
Code accompanying "Dynamic Neural Relational Inference" from CVPR 2020

Code accompanying "Dynamic Neural Relational Inference" This codebase accompanies the paper "Dynamic Neural Relational Inference" from CVPR 2020. This

Colin Graber 48 Dec 23, 2022
DANet for Tabular data classification/ regression.

Deep Abstract Networks A pyTorch implementation for AAAI-2022 paper DANets: Deep Abstract Networks for Tabular Data Classification and Regression. Bri

Ronnie Rocket 55 Sep 14, 2022
Official Implementation of SWAD (NeurIPS 2021)

SWAD: Domain Generalization by Seeking Flat Minima (NeurIPS'21) Official PyTorch implementation of SWAD: Domain Generalization by Seeking Flat Minima.

Junbum Cha 97 Dec 20, 2022
Improving the robustness and performance of biomedical NLP models through adversarial training

RobustBioNLP Improving the robustness and performance of biomedical NLP models through adversarial training In this repository you can find suppliment

Milad Moradi 3 Sep 20, 2022
Deeper DCGAN with AE stabilization

AEGeAN Deeper DCGAN with AE stabilization Parallel training of generative adversarial network as an autoencoder with dedicated losses for each stage.

Tyler Kvochick 36 Feb 17, 2022
Mscp jamf - Build compliance in jamf

mscp_jamf Build compliance in Jamf. This will build the following xml pieces to

Bob Gendler 3 Jul 25, 2022
TensorRT examples (Jetson, Python/C++)(object detection)

TensorRT examples (Jetson, Python/C++)(object detection)

Nobuo Tsukamoto 53 Dec 22, 2022
TensorFlow implementation of "TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?"

TokenLearner: What Can 8 Learned Tokens Do for Images and Videos? Source: Improving Vision Transformer Efficiency and Accuracy by Learning to Tokenize

Aritra Roy Gosthipaty 23 Dec 24, 2022