HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Related tags

Deep LearningHPRNet
Overview

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation

Official PyTroch implementation of HPRNet.

HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation,
Nermin Samet, Emre Akbas,
Under review. (arXiv pre-print)

Highlights

  • HPRNet is a bottom-up, one-stage and hierarchical keypoint regression method for whole-body pose estimation.
  • HPRNet has the best performance among bottom-up methods for all the whole-body parts.
  • HPRNet achieves SOTA performance for the face (76.0 AP) and hand (51.2 AP) keypoint estimation.
  • Unlike two-stage methods, HPRNet predicts whole-body pose in a constant time independent of the number of people in an image.

COCO-WholeBody Keypoint Estimation Results

Model Body AP Foot AP Face AP Hand AP Whole-body AP Download
HPRNet (DLA) 55.2 / 57.1 49.1 / 50.7 74.6 / 75.4 47.0 / 48.4 31.5 / 32.7 model
HPRNet (Hourglass) 59.4 / 61.1 53.0 / 53.9 75.4 / 76.0 50.4 / 51.2 34.8 / 34.9 model
  • Results are presented without and with test time flip augmentation respectively.
  • All models are trained on COCO-WholeBody train2017 and evaluated on val2017.
  • The models can be downloaded directly from Google drive.

Installation

  1. [Optional but recommended] create a new conda environment.

    conda create --name HPRNet python=3.7
    

    And activate the environment.

    conda activate HPRNet
    
  2. Clone the repo:

    HPRNet_ROOT=/path/to/clone/HPRNet
    git clone https://github.com/nerminsamet/HPRNet $HPRNet_ROOT
    
  3. Install PyTorch 1.4.0:

    conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
    
  4. Install the requirements:

    pip install -r requirements.txt
    
  5. Compile DCNv2 (Deformable Convolutional Networks):

    cd $HPRNet_ROOT/src/lib/models/networks/DCNv2
    ./make.sh
    

Dataset preparation

  • Download the images (2017 Train, 2017 Val) from coco website.

  • Download train and val annotation files.

    ${COCO_PATH}
    |-- annotations
        |-- coco_wholebody_train_v1.0.json
        |-- coco_wholebody_val_v1.0.json
    |-- images
        |-- train2017
        |-- val2017 
    

Evaluation and Training

  • You could find all the evaluation and training scripts in the experiments folder.
  • For evaluation, please download the pretrained models you want to evaluate and put them in HPRNet_ROOT/models/.
  • In the case that you don't have 4 GPUs, you can follow the linear learning rate rule to adjust the learning rate.
  • If the training is terminated before finishing, you can use the same command with --resume to resume training.

Acknowledgement

The numerical calculations reported in this paper were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).

License

HPRNet is released under the MIT License (refer to the LICENSE file for details).

Citation

If you find HPRNet useful for your research, please cite our paper as follows:

N. Samet, E. Akbas, "HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation", arXiv, 2021.

BibTeX entry:

@misc{hprnet,
      title={HPRNet: Hierarchical Point Regression for Whole-Body Human Pose Estimation}, 
      author={Nermin Samet and Emre Akbas},
      year={2021}, 
}
Owner
Nermin Samet
PhD candidate
Nermin Samet
Python package provinding tools for artistic interactive applications using AI

Documentation redrawing Python package provinding tools for artistic interactive applications using AI Created by ReDrawing Campinas team for the Open

ReDrawing Campinas 1 Sep 30, 2021
Code release for the paper “Worldsheet Wrapping the World in a 3D Sheet for View Synthesis from a Single Image”, ICCV 2021.

Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image This repository contains the code for the following paper: R. Hu,

Meta Research 37 Jan 04, 2023
A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Tensorpack is a neural network training interface based on TensorFlow. Features: It's Yet Another TF high-level API, with speed, and flexibility built

Tensorpack 6.2k Jan 09, 2023
This is Unofficial Repo. Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection (CVPR 2021)

Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection This is a PyTorch implementation of the LipForensics paper. This is an U

Minha Kim 2 May 11, 2022
Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

CoCa - Pytorch Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch. They were able to elegantly fit in contras

Phil Wang 565 Dec 30, 2022
Data and code from COVID-19 machine learning paper

Machine learning approaches for localized lockdown, subnotification analysis and cases forecasting in São Paulo state counties during COVID-19 pandemi

Sara Malvar 4 Dec 22, 2022
113 Nov 28, 2022
Image Captioning using CNN ,LSTM and Attention

Image Captioning using CNN ,LSTM and Attention This is a deeplearning model which tries to summarize an image into a text . Installation Install this

ASUTOSH GHANTO 1 Dec 16, 2021
SegNet model implemented using keras framework

keras-segnet Implementation of SegNet-like architecture using keras. Current version doesn't support index transferring proposed in SegNet article, so

185 Aug 30, 2022
Pytorch Lightning Distributed Accelerators using Ray

Distributed PyTorch Lightning Training on Ray This library adds new PyTorch Lightning plugins for distributed training using the Ray distributed compu

167 Jan 02, 2023
SemEval2022 Patronizing and Condescending Language (PCL) Detection

SemEval2022 Patronizing and Condescending Language (PCL) Detection This task is from SemEval 2022. What is Patronizing and Condescending Language (PCL

Daniel Saeedi 0 Aug 05, 2022
Code for database and frontend of webpage for Neural Fields in Visual Computing and Beyond.

Neural Fields in Visual Computing—Complementary Webpage This is based on the amazing MiniConf project from Hendrik Strobelt and Sasha Rush—thank you!

Brown University Visual Computing Group 29 Nov 30, 2022
Official Pytorch implementation of Meta Internal Learning

Official Pytorch implementation of Meta Internal Learning

10 Aug 24, 2022
Dilated Convolution for Semantic Image Segmentation

Multi-Scale Context Aggregation by Dilated Convolutions Introduction Properties of dilated convolution are discussed in our ICLR 2016 conference paper

Fisher Yu 764 Dec 26, 2022
Sketch-Based 3D Exploration with Stacked Generative Adversarial Networks

pix2vox [Demonstration video] Sketch-Based 3D Exploration with Stacked Generative Adversarial Networks. Generated samples Single-category generation M

Takumi Moriya 232 Nov 14, 2022
TransNet V2: Shot Boundary Detection Neural Network

TransNet V2: Shot Boundary Detection Neural Network This repository contains code for TransNet V2: An effective deep network architecture for fast sho

Tomáš Souček 212 Dec 27, 2022
Code release for NeX: Real-time View Synthesis with Neural Basis Expansion

NeX: Real-time View Synthesis with Neural Basis Expansion Project Page | Video | Paper | COLAB | Shiny Dataset We present NeX, a new approach to novel

536 Dec 20, 2022
SAMO: Streaming Architecture Mapping Optimisation

SAMO: Streaming Architecture Mapping Optimiser The SAMO framework provides a method of optimising the mapping of a Convolutional Neural Network model

Alexander Montgomerie-Corcoran 20 Dec 10, 2022
This repository allows the user to automatically scale a 3D model/mesh/point cloud on Agisoft Metashape

Metashape-Utils This repository allows the user to automatically scale a 3D model/mesh/point cloud on Agisoft Metashape, given a set of 2D coordinates

INSCRIBE 4 Nov 07, 2022
Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift"

Dangers of Bayesian Model Averaging under Covariate Shift This repository contains the code to reproduce the experiments in the paper Dangers of Bayes

Pavel Izmailov 25 Sep 21, 2022