The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".

Overview

This code was written during the course of my Bachelor thesis Classification of Human Whole-Body Motion using Hidden Markov Models. Some things might be broken and I definitely don't recommend to use any of the code in any sort of production application. However, for research purposes this code might be useful so I decided to open-source it. Use at your own risk!

Requirements

Use pip to install most requriements (pip install -r requriements.txt). Sometimes this causes problems if Cython, numpy and scipy are not already installed, in which case this needs to be done manually.

Additionally, some packages must be installed that are not provided by pip.

pySimox and pyMMM

pySimox and pyMMM must be installed manually as well. To build them, perform the following steps:

git submodule update --init --recursive
cd vendor/pySimox/build
cmake ..
make
cp _pysimox.so ../../../lib/python2.7/site-packages/_pysimox.so
cp pysimox.py ../../../lib/python2.7/site-packages/pysimox.py
cd ../pyMMM/build
cmake ..
make
cp _pymmm.so ../../../lib/python2.7/site-packages/_pymmm.so
cp pymmm.py ../../../lib/python2.7/site-packages/pymmm.py

Note that the installation script may need some fine-tuning. Additionally, this assumes that all virtualenv is set up in the root of this git repo.

Basic Usage

This repo contains two main programs: dataset.py and evaluate_new.py. All of them are located in src and should be run from this directory. There are some additional files in there, some of them are out-dated and should be deleted (e.g. evaluate.py), some of them are really just scripts and should be moved to the scripts folder eventually.

The dataset tool

The dataset tool is concerened with handling everything related to datasets: plot plots features, export saves a dataset in a variety of formats, report prints details about a dataset and check performs a consistency check. Additionally, export-all can be used to create a dataset that contains all features (normalized and unnormalized) by merging Vicon C3D and MMM files into one giant file. A couple of examples:

  • python dataset.py ../data/dataset1.json plot --features root_pos plots the root_pos feature of all motions in the dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json export --output ~/export.pkl exports dataset1 as a single pickled file; usually a JSON manifest is used
  • python dataset.py ../data/dataset1.json export-all --output ~/export_all.pkl exports dataset1 by combining vicon and MMM files and by computing both the normalized and unnormalized version of all features. It also performs normalization on the vicon data by using additional information from the MMM data (namely the root_pos and root_rot); the dataset has to be a JSON manifest
  • python dataset.py ../data/dataset1.json report prints details about a dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json check performs a consistency check of a dataset; the manifest has to be a JSON manifest

Additional parameters are avaialble for most commands. Use dataset --help to get an overview.

The evaluate_new tool

The evaluate_new tool can be used to perform feature selection (using the feature command) or to evaluate different types of models with decision makers (by using the model command). It is important to note that the evaluate_new tool expects a pickled version of the dataset, hence export or export_all must be used to prepare a dataset. This is to avoid the computational complexity.

A couple of examples:

  • python evaluate_new.py model ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --decision-maker log-regression --n-states 5 --model fhmm-seq --output-dir ~/out trains a HMM ensemble with each HMM having 5 states on the normalized_joint_pos and normalized_root_pos features and uses logistic regression to perform the final predicition. The results are also saved in the directory ~/out
  • python evaluate_new.py features ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --measure wasserstein performs feature selection using the starting set normalized_joint_pos normalized_root_pos and the wasserstein measure

From dataset to result

First, define a JSON manifest dataset.json that links together the individual motions and pick labels. Next, export the dataset by using python dataset.py ../data/dataset.json export-all --output ../data/dataset_all.pkl. If you need smoothing, simply load the dataset (using pickle.load()), call smooth_features() on the Dataset object and dump it to a new file. There's currently no script for this but it can be done using three lines and the interactive python interpreter. Next, perform feature selection using python evaluate_new.py features ../data/dataset_all.pkl --features <list of features> --measure wasserstein --output-dir ~/features --transformers minmax-scaler. You'll want to use the minmax scaler transformer to avoid numerical problems during training. This will probably take a while. The results (at ~/features) will give you the best feature subsets that were found. Next, use those features to train an HMM ensemble: python evaluate_new model ../data/dataset_all.pkl --features <best features> --model fhmm-seq --n-chains 2 --n-states 10 --n-training-iter 30 -decision-maker log-regression --transformers minmax-scaler --output-dir ~/train (again, the minmax-scaler is almost always a good idea). The results will be in ~/output.

Owner
Matthias Plappert
I am a research scientist working on machine learning, and especially deep reinforcement learning, in robotics.
Matthias Plappert
Towards Boosting the Accuracy of Non-Latin Scene Text Recognition

Convolutional Recurrent Neural Network + CTCLoss | STAR-Net Code for paper "Towards Boosting the Accuracy of Non-Latin Scene Text Recognition" Depende

Sanjana Gunna 7 Aug 07, 2022
A module for solving and visualizing Schrödinger equation.

qmsolve This is an attempt at making a solid, easy to use solver, capable of solving and visualize the Schrödinger equation for multiple particles, an

506 Dec 28, 2022
Code for the paper "Reinforced Active Learning for Image Segmentation"

Reinforced Active Learning for Image Segmentation (RALIS) Code for the paper Reinforced Active Learning for Image Segmentation Dependencies python 3.6

Arantxa Casanova 79 Dec 19, 2022
Second Order Optimization and Curvature Estimation with K-FAC in JAX.

KFAC-JAX - Second Order Optimization with Approximate Curvature in JAX Installation | Quickstart | Documentation | Examples | Citing KFAC-JAX KFAC-JAX

DeepMind 90 Dec 22, 2022
Labels4Free: Unsupervised Segmentation using StyleGAN

Labels4Free: Unsupervised Segmentation using StyleGAN ICCV 2021 Figure: Some segmentation masks predicted by Labels4Free Framework on real and synthet

70 Dec 23, 2022
Medical image analysis framework merging ANTsPy and deep learning

ANTsPyNet A collection of deep learning architectures and applications ported to the python language and tools for basic medical image processing. Bas

Advanced Normalization Tools Ecosystem 118 Dec 24, 2022
A fuzzing framework for SMT solvers

yinyang A fuzzing framework for SMT solvers. Given a set of seed SMT formulas, yinyang generates mutant formulas to stress-test SMT solvers. yinyang c

Project Yin-Yang for SMT Solver Testing 145 Jan 04, 2023
A certifiable defense against adversarial examples by training neural networks to be provably robust

DiffAI v3 DiffAI is a system for training neural networks to be provably robust and for proving that they are robust. The system was developed for the

SRI Lab, ETH Zurich 202 Dec 13, 2022
A deep learning network built with TensorFlow and Keras to classify gender and estimate age.

Convolutional Neural Network (CNN). This repository contains a source code of a deep learning network built with TensorFlow and Keras to classify gend

Pawel Dziemiach 1 Dec 18, 2021
Buffon’s needle: one of the oldest problems in geometric probability

Buffon-s-Needle Buffon’s needle is one of the oldest problems in geometric proba

3 Feb 18, 2022
PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference

PyTorch implementation of [1611.06440 Pruning Convolutional Neural Networks for Resource Efficient Inference] This demonstrates pruning a VGG16 based

Jacob Gildenblat 836 Dec 26, 2022
Official repository for Jia, Raghunathan, Göksel, and Liang, "Certified Robustness to Adversarial Word Substitutions" (EMNLP 2019)

Certified Robustness to Adversarial Word Substitutions This is the official GitHub repository for the following paper: Certified Robustness to Adversa

Robin Jia 38 Oct 16, 2022
MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

Microsoft 5.7k Jan 09, 2023
Covid19-Forecasting - An interactive website that tracks, models and predicts COVID-19 Cases

Covid-Tracker This is an interactive website that tracks, models and predicts CO

Adam Lahmadi 1 Feb 01, 2022
A Multi-modal Model Chinese Spell Checker Released on ACL2021.

ReaLiSe ReaLiSe is a multi-modal Chinese spell checking model. This the office code for the paper Read, Listen, and See: Leveraging Multimodal Informa

DaDa 106 Dec 29, 2022
Efficient Two-Step Networks for Temporal Action Segmentation (Neurocomputing 2021)

Efficient Two-Step Networks for Temporal Action Segmentation This repository provides a PyTorch implementation of the paper Efficient Two-Step Network

8 Apr 16, 2022
Semi-automated OpenVINO benchmark_app with variable parameters

Semi-automated OpenVINO benchmark_app with variable parameters. User can specify multiple options for any parameters in the benchmark_app and the progam runs the benchmark with all combinations of gi

Yasunori Shimura 8 Apr 11, 2022
NeurIPS 2021, "Fine Samples for Learning with Noisy Labels"

[Official] FINE Samples for Learning with Noisy Labels This repository is the official implementation of "FINE Samples for Learning with Noisy Labels"

mythbuster 27 Dec 23, 2022
Semi-supervised Stance Detection of Tweets Via Distant Network Supervision

SANDS This is an annonymous repository containing code and data necessary to reproduce the results published in "Semi-supervised Stance Detection of T

2 Sep 22, 2022
Publication describing 3 ML examples at NSLS-II and interfacing into Bluesky

Machine learning enabling high-throughput and remote operations at large-scale user facilities. Overview This repository contains the source code and

BNL 4 Sep 24, 2022