The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".

Overview

This code was written during the course of my Bachelor thesis Classification of Human Whole-Body Motion using Hidden Markov Models. Some things might be broken and I definitely don't recommend to use any of the code in any sort of production application. However, for research purposes this code might be useful so I decided to open-source it. Use at your own risk!

Requirements

Use pip to install most requriements (pip install -r requriements.txt). Sometimes this causes problems if Cython, numpy and scipy are not already installed, in which case this needs to be done manually.

Additionally, some packages must be installed that are not provided by pip.

pySimox and pyMMM

pySimox and pyMMM must be installed manually as well. To build them, perform the following steps:

git submodule update --init --recursive
cd vendor/pySimox/build
cmake ..
make
cp _pysimox.so ../../../lib/python2.7/site-packages/_pysimox.so
cp pysimox.py ../../../lib/python2.7/site-packages/pysimox.py
cd ../pyMMM/build
cmake ..
make
cp _pymmm.so ../../../lib/python2.7/site-packages/_pymmm.so
cp pymmm.py ../../../lib/python2.7/site-packages/pymmm.py

Note that the installation script may need some fine-tuning. Additionally, this assumes that all virtualenv is set up in the root of this git repo.

Basic Usage

This repo contains two main programs: dataset.py and evaluate_new.py. All of them are located in src and should be run from this directory. There are some additional files in there, some of them are out-dated and should be deleted (e.g. evaluate.py), some of them are really just scripts and should be moved to the scripts folder eventually.

The dataset tool

The dataset tool is concerened with handling everything related to datasets: plot plots features, export saves a dataset in a variety of formats, report prints details about a dataset and check performs a consistency check. Additionally, export-all can be used to create a dataset that contains all features (normalized and unnormalized) by merging Vicon C3D and MMM files into one giant file. A couple of examples:

  • python dataset.py ../data/dataset1.json plot --features root_pos plots the root_pos feature of all motions in the dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json export --output ~/export.pkl exports dataset1 as a single pickled file; usually a JSON manifest is used
  • python dataset.py ../data/dataset1.json export-all --output ~/export_all.pkl exports dataset1 by combining vicon and MMM files and by computing both the normalized and unnormalized version of all features. It also performs normalization on the vicon data by using additional information from the MMM data (namely the root_pos and root_rot); the dataset has to be a JSON manifest
  • python dataset.py ../data/dataset1.json report prints details about a dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json check performs a consistency check of a dataset; the manifest has to be a JSON manifest

Additional parameters are avaialble for most commands. Use dataset --help to get an overview.

The evaluate_new tool

The evaluate_new tool can be used to perform feature selection (using the feature command) or to evaluate different types of models with decision makers (by using the model command). It is important to note that the evaluate_new tool expects a pickled version of the dataset, hence export or export_all must be used to prepare a dataset. This is to avoid the computational complexity.

A couple of examples:

  • python evaluate_new.py model ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --decision-maker log-regression --n-states 5 --model fhmm-seq --output-dir ~/out trains a HMM ensemble with each HMM having 5 states on the normalized_joint_pos and normalized_root_pos features and uses logistic regression to perform the final predicition. The results are also saved in the directory ~/out
  • python evaluate_new.py features ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --measure wasserstein performs feature selection using the starting set normalized_joint_pos normalized_root_pos and the wasserstein measure

From dataset to result

First, define a JSON manifest dataset.json that links together the individual motions and pick labels. Next, export the dataset by using python dataset.py ../data/dataset.json export-all --output ../data/dataset_all.pkl. If you need smoothing, simply load the dataset (using pickle.load()), call smooth_features() on the Dataset object and dump it to a new file. There's currently no script for this but it can be done using three lines and the interactive python interpreter. Next, perform feature selection using python evaluate_new.py features ../data/dataset_all.pkl --features <list of features> --measure wasserstein --output-dir ~/features --transformers minmax-scaler. You'll want to use the minmax scaler transformer to avoid numerical problems during training. This will probably take a while. The results (at ~/features) will give you the best feature subsets that were found. Next, use those features to train an HMM ensemble: python evaluate_new model ../data/dataset_all.pkl --features <best features> --model fhmm-seq --n-chains 2 --n-states 10 --n-training-iter 30 -decision-maker log-regression --transformers minmax-scaler --output-dir ~/train (again, the minmax-scaler is almost always a good idea). The results will be in ~/output.

Owner
Matthias Plappert
I am a research scientist working on machine learning, and especially deep reinforcement learning, in robotics.
Matthias Plappert
PINN Burgers - 1D Burgers equation simulated by PINN

PINN(s): Physics-Informed Neural Network(s) for Burgers equation This is an impl

ShotaDEGUCHI 1 Feb 12, 2022
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

AugMix Introduction We propose AugMix, a data processing technique that mixes augmented images and enforces consistent embeddings of the augmented ima

Google Research 876 Dec 17, 2022
A Closer Look at Reference Learning for Fourier Phase Retrieval

A Closer Look at Reference Learning for Fourier Phase Retrieval This repository contains code for our NeurIPS 2021 Workshop on Deep Learning and Inver

Tobias Uelwer 1 Oct 28, 2021
Code for intrusion detection system (IDS) development using CNN models and transfer learning

Intrusion-Detection-System-Using-CNN-and-Transfer-Learning This is the code for the paper entitled "A Transfer Learning and Optimized CNN Based Intrus

Western OC2 Lab 38 Dec 12, 2022
Automatic 2D-to-3D Video Conversion with CNNs

Deep3D: Automatic 2D-to-3D Video Conversion with CNNs How To Run To run this code. Please install MXNet following the official document. Deep3D requir

Eric Junyuan Xie 1.2k Dec 30, 2022
Official PyTorch Implementation of "Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs". NeurIPS 2020.

Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs This repository is the implementation of SELAR. Dasol Hwang* , Jinyoung Pa

MLV Lab (Machine Learning and Vision Lab at Korea University) 48 Nov 09, 2022
Computing Shapley values using VAEAC

Shapley values and the VAEAC method In this GitHub repository, we present the implementation of the VAEAC approach from our paper "Using Shapley Value

3 Nov 23, 2022
4D Human Body Capture from Egocentric Video via 3D Scene Grounding

4D Human Body Capture from Egocentric Video via 3D Scene Grounding [Project] [Paper] Installation: Our method requires the same dependencies as SMPLif

Miao Liu 37 Nov 08, 2022
A modular PyTorch library for optical flow estimation using neural networks

A modular PyTorch library for optical flow estimation using neural networks

neu-vig 113 Dec 20, 2022
PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech

PortaSpeech - PyTorch Implementation PyTorch Implementation of PortaSpeech: Portable and High-Quality Generative Text-to-Speech. Model Size Module Nor

Keon Lee 279 Jan 04, 2023
This is a simple backtesting framework to help you test your crypto currency trading. It includes a way to download and store historical crypto data and to execute a trading strategy.

You can use this simple crypto backtesting script to ensure your trading strategy is successful Minimal setup required and works well with static TP a

Andrei 154 Sep 12, 2022
Fuzzing tool (TFuzz): a fuzzing tool based on program transformation

T-Fuzz T-Fuzz consists of 2 components: Fuzzing tool (TFuzz): a fuzzing tool based on program transformation Crash Analyzer (CrashAnalyzer): a tool th

HexHive 244 Nov 09, 2022
PyTorch implementation of image classification models for CIFAR-10/CIFAR-100/MNIST/FashionMNIST/Kuzushiji-MNIST/ImageNet

PyTorch Image Classification Following papers are implemented using PyTorch. ResNet (1512.03385) ResNet-preact (1603.05027) WRN (1605.07146) DenseNet

1.2k Jan 04, 2023
Second-Order Neural ODE Optimizer, NeurIPS 2021 spotlight

Second-order Neural ODE Optimizer (NeurIPS 2021 Spotlight) [arXiv] ✔️ faster convergence in wall-clock time | ✔️ O(1) memory cost | ✔️ better test-tim

Guan-Horng Liu 39 Oct 22, 2022
Code + pre-trained models for the paper Keeping Your Eye on the Ball Trajectory Attention in Video Transformers

Motionformer This is an official pytorch implementation of paper Keeping Your Eye on the Ball: Trajectory Attention in Video Transformers. In this rep

Facebook Research 192 Dec 23, 2022
Custom implementation of Corrleation Module

Pytorch Correlation module this is a custom C++/Cuda implementation of Correlation module, used e.g. in FlowNetC This tutorial was used as a basis for

Clément Pinard 361 Dec 12, 2022
git《Tangent Space Backpropogation for 3D Transformation Groups》(CVPR 2021) GitHub:1]

LieTorch: Tangent Space Backpropagation Introduction The LieTorch library generalizes PyTorch to 3D transformation groups. Just as torch.Tensor is a m

Princeton Vision & Learning Lab 482 Jan 06, 2023
Code to train models from "Paraphrastic Representations at Scale".

Paraphrastic Representations at Scale Code to train models from "Paraphrastic Representations at Scale". The code is written in Python 3.7 and require

John Wieting 71 Dec 19, 2022
Official repository with code and data accompanying the NAACL 2021 paper "Hurdles to Progress in Long-form Question Answering" (https://arxiv.org/abs/2103.06332).

Hurdles to Progress in Long-form Question Answering This repository contains the official scripts and datasets accompanying our NAACL 2021 paper, "Hur

Kalpesh Krishna 41 Nov 08, 2022
Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds This is the official code implementation for the paper "Spatio-temporal Se

Hesper 63 Jan 05, 2023