CTR-GCN
This repo is the official implementation for Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition. The paper is accepted to ICCV2021.
Note: We also provide a simple and strong baseline model, which achieves 83.7% on NTU120 CSub with joint modality only, to facilitate the development of skeleton-based action recognition.
Architecture of CTR-GC
Prerequisites
-
Python >= 3.6
-
PyTorch >= 1.1.0
-
PyYAML, tqdm, tensorboardX
-
We provide the dependency file of our experimental environment, you can install all dependencies by creating a new anaconda virtual environment and running
pip install -r requirements.txt -
Run
pip install -e torchlight
Data Preparation
Download datasets.
There are 3 datasets to download:
- NTU RGB+D 60 Skeleton
- NTU RGB+D 120 Skeleton
- NW-UCLA
NTU RGB+D 60 and 120
- Request dataset here: http://rose1.ntu.edu.sg/Datasets/actionRecognition.asp
- Download the skeleton-only datasets:
nturgbd_skeletons_s001_to_s017.zip(NTU RGB+D 60)nturgbd_skeletons_s018_to_s032.zip(NTU RGB+D 120)- Extract above files to
./data/nturgbd_raw
NW-UCLA
- Download dataset from here
- Move
all_sqeto./data/NW-UCLA
Data Processing
Directory Structure
Put downloaded data into the following directory structure:
- data/
- NW-UCLA/
- all_sqe
... # raw data of NW-UCLA
- ntu/
- ntu120/
- nturgbd_raw/
- nturgb+d_skeletons/ # from `nturgbd_skeletons_s001_to_s017.zip`
...
- nturgb+d_skeletons120/ # from `nturgbd_skeletons_s018_to_s032.zip`
...
Generating Data
- Generate NTU RGB+D 60 or NTU RGB+D 120 dataset:
cd ./data/ntu # or cd ./data/ntu120
# Get skeleton of each performer
python get_raw_skes_data.py
# Remove the bad skeleton
python get_raw_denoised_data.py
# Transform the skeleton to the center of the first frame
python seq_transformation.py
Training & Testing
Training
- Change the config file depending on what you want.
# Example: training CTRGCN on NTU RGB+D 120 cross subject with GPU 0
python main.py --config config/nturgbd120-cross-subject/default.yaml --work-dir work_dir/ntu120/csub/ctrgcn --device 0
# Example: training provided baseline on NTU RGB+D 120 cross subject
python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.baseline.Model--work-dir work_dir/ntu120/csub/baseline --device 0
- To train model on NTU RGB+D 60/120 with bone or motion modalities, setting
boneorvelarguments in the config filedefault.yamlor in the command line.
# Example: training CTRGCN on NTU RGB+D 120 cross subject under bone modality
python main.py --config config/nturgbd120-cross-subject/default.yaml --train_feeder_args bone=True --test_feeder_args bone=True --work-dir work_dir/ntu120/csub/ctrgcn_bone --device 0
- To train model on NW-UCLA with bone or motion modalities, you need to modify
data_pathintrain_feeder_argsandtest_feeder_argsto "bone" or "motion" or "bone motion", and run
python main.py --config config/ucla/default.yaml --work-dir work_dir/ucla/ctrgcn_xxx --device 0
- To train your own model, put model file
your_model.pyunder./modeland run:
# Example: training your own model on NTU RGB+D 120 cross subject
python main.py --config config/nturgbd120-cross-subject/default.yaml --model model.your_model.Model --work-dir work_dir/ntu120/csub/your_model --device 0
Testing
- To test the trained models saved in <work_dir>, run the following command:
python main.py --config <work_dir>/config.yaml --work-dir <work_dir> --phase test --save-score True --weights <work_dir>/xxx.pt --device 0
- To ensemble the results of different modalities, run
# Example: ensemble four modalities of CTRGCN on NTU RGB+D 120 cross subject
python ensemble.py --datasets ntu120/xsub --joint-dir work_dir/ntu120/csub/ctrgcn --bone-dir work_dir/ntu120/csub/ctrgcn_bone --joint-motion-dir work_dir/ntu120/csub/ctrgcn_motion --bone-motion-dir work_dir/ntu120/csub/ctrgcn_bone_motion
Pretrained Models
- Download pretrained models for producing the final results on NTU RGB+D 60&120 cross subject [Google Drive].
- Put files to <work_dir> and run Testing command to produce the final result.
Acknowledgements
This repo is based on 2s-AGCN. The data processing is borrowed from SGN and HCN.
Thanks to the original authors for their work!
Citation
Please cite this work if you find it useful:.
@article{chen2021channel,
title={Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition},
author={Chen, Yuxin and Zhang, Ziqi and Yuan, Chunfeng and Li, Bing and Deng, Ying and Hu, Weiming},
journal={arXiv preprint arXiv:2107.12213},
year={2021}
}
Contact
For any questions, feel free to contact: [email protected]
3 Jun 22, 2022
18 Oct 12, 2022
16 Aug 30, 2022
35 Aug 21, 2022
264 Jan 09, 2023
140 Dec 07, 2022
66 Nov 24, 2022
100 Nov 30, 2022
76 Dec 20, 2022
26 Dec 02, 2022
59 Dec 14, 2022
5 Sep 05, 2022
55 Dec 12, 2022
2 Aug 07, 2022
8 May 20, 2022
359 Jan 01, 2023
8 Aug 31, 2022
683 Dec 28, 2022
1.4k Dec 30, 2022
3 Oct 07, 2022