Repo for "Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks"

Overview

Summary

This is the code for the paper Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks by Yanxiang Wang, Xian Zhang, Yiran Shen*, Bowen Du, Guangrong Zhao, Lizhen Cui Cui Lizhen, Hongkai Wen.

The paper can be found here.

Introduction

In this paper, We propose new event-based gait recognition approaches basing on two different representations of the event-stream, i.e., graph and image-like representations, and use Graph-based Convolutional Network (GCN) and Convolutional Neural Networks (CNN) respectively to recognize gait from the event-streams. The two approaches are termed as EV-Gait-3DGraph and EV-Gait-IMG. To evaluate the performance of the proposed approaches, we collect two event-based gait datasets, one from real-world experiments and the other by converting the publicly available RGB gait recognition benchmark CASIA-B.

If you use any of this code or data, please cite the following publication:

@inproceedings{wang2019ev,
  title={EV-gait: Event-based robust gait recognition using dynamic vision sensors},
  author={Wang, Yanxiang and Du, Bowen and Shen, Yiran and Wu, Kai and Zhao, Guangrong and Sun, Jianguo and Wen, Hongkai},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={6358--6367},
  year={2019}
}
@article{wang2021event,
 title={Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks},
    author={Wang, Yanxiang and Zhang, Xian and Shen, Yiran and Du, Bowen and Zhao,     Guangrong and Lizhen, Lizhen Cui Cui and Wen, Hongkai},
   journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    year={2021},
   publisher={IEEE}
   }

Requirements

  • Python 3.x
  • Conda
  • cuda
  • PyTorch
  • numpy
  • scipy
  • PyTorch Geometric
  • TensorFlow
  • Matlab (with Computer Vision Toolbox and Image Processing Toolbox for nonuniform grid downsample)

Installation

Data

We use both data collected in real-world experiments(called DVS128-Gait) and converted from publicly available RGB gait databases(called EV-CASIA-B). Here we offer the code and data for the DVS128-Gait.

DVS128-Gait DATASET

we use a DVS128 Dynamic Vision Sensor from iniVation operating at 128*128 pixel resolution.

we collect two dataset: DVS128-Gait-Day and DVS128-Gait-Night, which were collected under day and night lighting condition respectively.

For each lighting condition, we recruited 20 volunteers to contribute their data in two experiment sessions spanning over a few days. In each session, the participants were asked to repeat walking in front of the DVS128 sensor for 100 times.

Run EV-Gait-3DGraph

  • download DVS128-Gait-Day dataset, you will get DVS128-Gait-Day folder which contains train and test data, place DVS128-Gait-Day folder to the data/ folder.

  • event downsample using matlab:

    1. open Matlab
    2. go to matlab_downsample
    3. run main.m. This will generate the data/DVS128-Gait-Day/downsample folder which contains the non-uniform octreeGrid filtering data .
  • or directly download the downsampled data from this link:

    https://pan.baidu.com/s/1OKKvrhid929DakSxsjT7XA , extraction code: ceb1

    Then unzip it to the data/DVS128-Gait-Day/downsample folder.

  • generate graph representation for event, the graph data will be generated in data/DVS128-Gait-Day/graph folder:

    cd generate_graph
    python mat2graph.py
    
  • Download the pretrained model to the trained_model folder:

    https://pan.baidu.com/s/1X7eytUDWAtKS4bk0rjbs6g , extraction code: b7z7

  • run EV-Gait-3DGraph model with the pretrained model:

    cd EV-Gait-3DGraph
    python test_3d_graph.py --model_name EV_Gait_3DGraph.pkl
    

    The parameter--model_name refers to the downloaded pretrained model name.

  • train EV-Gait-3DGraph from scratch:

    cd EV-Gait-3DGraph
    nohup python -u train_3d_graph.py --epoch 110 --cuda 0 > train_3d_graph.log 2>&1 &
    

    the traning log would be created at log/train.log.

    parameters of train_3d_graph.py

    • --batch_size: default 16
    • --epoch: number of iterations, default 150
    • --cuda: specify the cuda device to use, default 0

Run EV-Gait-IMG

  • generate the image-like representation

    cd EV-Gait-IMG
    python make_hdf5.py
    
  • Download the pretrained model to the trained_model folder:

    https://pan.baidu.com/s/1xNbYUYYVPTwwjXeQABjmUw , extraction code: g5k2

    we provide four well trained model for four image-like representations presented in the paper.

    • EV_Gait_IMG_four_channel.pkl
    • EV_Gait_IMG_counts_only_two_channel.pkl
    • EV_Gait_IMG_time_only_two_channel.pkl
    • EV_Gait_IMG_counts_and_time_two_channel.pkl
  • run EV-Gait-IMG model with the pretrained model:

    We provide four options for --img_type to correctly test the corresponding image-like representation

    • four_channel : All four channels are considered, which is the original setup of the image-like representation

      python test_gait_cnn.py --img_type four_channel --model_name EV_Gait_IMG_four_channel.pkl
      
    • counts_only_two_channel : Only the two channels accommodating the counts of positive or negative events are kept

      python test_gait_cnn.py --img_type counts_only_two_channel --model_name EV_Gait_IMG_counts_only_two_channel.pkl
      
    • time_only_two_channel : Only the two channels holding temporal characteristics are kept

      python test_gait_cnn.py --img_type time_only_two_channel --model_name EV_Gait_IMG_time_only_two_channel.pkl
      
    • counts_and_time_two_channel : The polarity of the events is removed

      python test_gait_cnn.py --img_type counts_and_time_two_channel --model_name EV_Gait_IMG_counts_and_time_two_channel.pkl
      

    The parameter --model_name refers to the downloaded pretrained model name.

  • train EV-Gait-IMG from scratch:

    nohup python -u train_gait_cnn.py --img_type counts_only_two_channel --epoch 50 --cuda 1 --batch_size 128 > counts_only_two_channel.log 2>&1 &
    

    parameters of test_gait_cnn.py

    • --batch_size: default 128
    • --epoch: number of iterations, default 50
    • --cuda: specify the cuda device to use, default 0
    • --img_type: specify the type of image-like representation to train the cnn. Four options are provided according to the paper.
      • four_channel : All four channels are considered, which is the original setup of the image-like representation
      • counts_only_two_channel : Only the two channels accommodating the counts of positive or negative events are kept.
      • time_only_two_channel : Only the two channels holding temporal characteristics are kept.
      • counts_and_time_two_channel : The polarity of the events is removed.
Owner
zhangxian
Student
zhangxian
Exact Pareto Optimal solutions for preference based Multi-Objective Optimization

Exact Pareto Optimal solutions for preference based Multi-Objective Optimization

Debabrata Mahapatra 40 Dec 24, 2022
The challenge for Quantum Coalition Hackathon 2021

Qchack 2021 Google Challenge This is a challenge for the brave 2021 qchack.io participants. Instructions Hello, intrepid qchacker, welcome to the G|o

quantumlib 18 May 04, 2022
The AWS Certified SysOps Administrator

The AWS Certified SysOps Administrator – Associate (SOA-C02) exam is intended for system administrators in a cloud operations role who have at least 1 year of hands-on experience with deployment, man

Aiden Pearce 32 Dec 11, 2022
Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Constrained Logistic Regression Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (v

1 Dec 29, 2021
A PyTorch implementation of EfficientDet.

A PyTorch impl of EfficientDet faithful to the original Google impl w/ ported weights

Ross Wightman 1.4k Jan 07, 2023
AI Face Mesh: This is a simple face mesh detection program based on Artificial intelligence.

AI Face Mesh: This is a simple face mesh detection program based on Artificial Intelligence which made with Python. It's able to detect 468 different

Md. Rakibul Islam 1 Jan 13, 2022
Official PyTorch code for the paper: "Point-Based Modeling of Human Clothing" (ICCV 2021)

Point-Based Modeling of Human Clothing Paper | Project page | Video This is an official PyTorch code repository of the paper "Point-Based Modeling of

Visual Understanding Lab @ Samsung AI Center Moscow 64 Nov 22, 2022
Learning to Reach Goals via Iterated Supervised Learning

Vanilla GCSL This repository contains a vanilla implementation of "Learning to Reach Goals via Iterated Supervised Learning" proposed by Dibya Gosh et

Christoph Heindl 4 Aug 10, 2022
GANmouflage: 3D Object Nondetection with Texture Fields

GANmouflage: 3D Object Nondetection with Texture Fields Rui Guo1 Jasmine Collins

29 Aug 10, 2022
Improved Fitness Optimization Landscapes for Sequence Design

ReLSO Improved Fitness Optimization Landscapes for Sequence Design Description Citation How to run Training models Original data source Description In

Krishnaswamy Lab 44 Dec 20, 2022
Use tensorflow to implement a Deep Neural Network for real time lane detection

LaneNet-Lane-Detection Use tensorflow to implement a Deep Neural Network for real time lane detection mainly based on the IEEE IV conference paper "To

MaybeShewill-CV 1.9k Jan 08, 2023
DeepStochlog Package For Python

DeepStochLog Installation Installing SWI Prolog DeepStochLog requires SWI Prolog to run. Run the following commands to install: sudo apt-add-repositor

KU Leuven Machine Learning Research Group 17 Dec 23, 2022
A Lightweight Experiment & Resource Monitoring Tool 📺

Lightweight Experiment & Resource Monitoring 📺 "Did I already run this experiment before? How many resources are currently available on my cluster?"

170 Dec 28, 2022
Python package facilitating the use of Bayesian Deep Learning methods with Variational Inference for PyTorch

PyVarInf PyVarInf provides facilities to easily train your PyTorch neural network models using variational inference. Bayesian Deep Learning with Vari

342 Dec 02, 2022
上海交通大学全自动抢课脚本,支持准点开抢与抢课后持续捡漏两种模式。2021/06/08更新。

Welcome to Course-Bullying-in-SJTU-v3.1! 2021/6/8 紧急更新v3.1 更新说明 为了更好地保护用户隐私,将原来用户名+密码的登录方式改为微信扫二维码+cookie登录方式,不再需要配置使用pytesseract。在使用扫码登录模式时,请稍等,二维码将马

87 Sep 13, 2022
A simple software for capturing human body movements using the Kinect camera.

KinectMotionCapture A simple software for capturing human body movements using the Kinect camera. The software can seamlessly save joints and bones po

Aleksander Palkowski 5 Aug 13, 2022
Advanced Signal Processing Notebooks and Tutorials

Advanced Digital Signal Processing Notebooks and Tutorials Prof. Dr. -Ing. Gerald Schuller Jupyter Notebooks and Videos: Renato Profeta Applied Media

Guitars.AI 115 Dec 13, 2022
House_prices_kaggle - Predict sales prices and practice feature engineering, RFs, and gradient boosting

House Prices - Advanced Regression Techniques Predicting House Prices with Machine Learning This project is build to enhance my knowledge about machin

Gurpreet Singh 1 Jan 01, 2022
Sequence lineage information extracted from RKI sequence data repo

Pango lineage information for German SARS-CoV-2 sequences This repository contains a join of the metadata and pango lineage tables of all German SARS-

Cornelius Roemer 24 Oct 26, 2022
[CVPR 2020] Interpreting the Latent Space of GANs for Semantic Face Editing

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing Figure: High-quality facial attributes editing results with InterFaceGA

GenForce: May Generative Force Be with You 1.3k Dec 29, 2022