Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

Related tags

Deep LearningTTP
Overview

Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation

This is an official implementation of the NeurIPS 2021 paper: Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation. More details can be found at our project website.

teaser

Preparation

  1. Install dependencies
pip install -r requirements.txt
  1. Make libs

    cd ${PROJECT_ROOT}/lib
    make
  2. Place Penn Action data in data directory. (Instructions on Human3.6M and BBC Pose are coming soon.)

    Your directory tree should look like this:

    ${PROJECT_ROOT}
    └── data
        └── Penn_Action
            ├── frames
            ├── labels
            ├── tools
            └── README
    
  3. Download pretrained model of ResNet-18 and ResNet-50 and place them in models/pytorch/imagenet.

    Your directory tree should look like this:

    ${PROJECT_ROOT}
    └── models
        └── pytorch
            └── imagenet
                ├── resnet18-5c106cde.pth
                └── resnet50-19c8e357.pth
    

Training and Test-time Personalization

Training

python tools/train_joint.py \
   --cfg experiments/penn/joint_res50_128x128_1e-3_comb_attn_tf1_4head.yaml

Run Test-Time Personalization (online)

python tools/test_time_training.py \
   --cfg experiments/penn/ttp_res50_128x128_lr1e-4_online_downsample1_comb_attn_tf1_4head.yaml \
   TEST.MODEL_FILE ${MODEL_FILE}

Run Test-Time Personalization (offline)

python tools/test_time_training.py \
   --cfg experiments/penn/ttp_res50_128x128_lr1e-4_offline_downsample1_comb_attn_tf1_4head.yaml \
   TEST.MODEL_FILE ${MODEL_FILE}

Baseline Model

To train the baseline model for comparison

python tools/train.py --cfg experiments/penn/res50_128x128.yaml

Result

Configs, results and model checkpoints on Human3.6M and BBC Pose are coming soon.

Method TTP Scenario Penn Action Checkpoint
Baseline - 85.233 Google Drive
Ours before TTP 86.283 Google Drive
Ours online 87.660 -
Ours offline 88.633 -

Acknowlegement

TTP is developed based on HRNet. We also incorperate some code from IMM.

On-device speech-to-index engine powered by deep learning.

On-device speech-to-index engine powered by deep learning.

Picovoice 30 Nov 24, 2022
Türkiye Canlı Mobese Görüntülerinde Profesyonel Nesne Takip Sistemi

Türkiye Mobese Görüntü Takip Türkiye Mobese görüntülerinde OPENCV ve Yolo ile takip sistemi Multiple Object Tracking System in Turkish Mobese with OPE

15 Dec 22, 2022
A Learning-based Camera Calibration Toolbox

Learning-based Camera Calibration A Learning-based Camera Calibration Toolbox Paper The pdf file can be found here. @misc{zhang2022learningbased,

Eason 14 Dec 21, 2022
Code of Periodic Activation Functions Induce Stationarity

Periodic Activation Functions Induce Stationarity This repository is the official implementation of the methods in the publication: L. Meronen, M. Tra

AaltoML 12 Jun 07, 2022
This is the pytorch implementation for the paper: *Learning Accurate Performance Predictors for Ultrafast Automated Model Compression*, which is in submission to TPAMI

SeerNet This is the pytorch implementation for the paper: Learning Accurate Performance Predictors for Ultrafast Automated Model Compression, which is

3 May 01, 2022
Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

ONNX Object Localization Network Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX. Ori

Ibai Gorordo 15 Oct 14, 2022
BERTMap: A BERT-Based Ontology Alignment System

BERTMap: A BERT-based Ontology Alignment System Important Notices The relevant paper was accepted in AAAI-2022. Arxiv version is available at: https:/

KRR 36 Dec 24, 2022
RetinaFace: Deep Face Detection Library in TensorFlow for Python

RetinaFace is a deep learning based cutting-edge facial detector for Python coming with facial landmarks.

Sefik Ilkin Serengil 512 Dec 29, 2022
Serverless proxy for Spark cluster

Hydrosphere Mist Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model f

hydrosphere.io 317 Dec 01, 2022
The official PyTorch implementation of Curriculum by Smoothing (NeurIPS 2020, Spotlight).

Curriculum by Smoothing (NeurIPS 2020) The official PyTorch implementation of Curriculum by Smoothing (NeurIPS 2020, Spotlight). For any questions reg

PAIR Lab 36 Nov 23, 2022
On the Analysis of French Phonetic Idiosyncrasies for Accent Recognition

On the Analysis of French Phonetic Idiosyncrasies for Accent Recognition With the spirit of reproducible research, this repository contains codes requ

0 Feb 24, 2022
Music Classification: Beyond Supervised Learning, Towards Real-world Applications

Music Classification: Beyond Supervised Learning, Towards Real-world Applications

104 Dec 15, 2022
Efficient face emotion recognition in photos and videos

This repository contains code of face emotion recognition that was developed in the RSF (Russian Science Foundation) project no. 20-71-10010 (Efficien

Andrey Savchenko 239 Jan 04, 2023
CowHerd is a partially-observed reinforcement learning environment

CowHerd is a partially-observed reinforcement learning environment, where the player walks around an area and is rewarded for milking cows. The cows try to escape and the player can place fences to h

Danijar Hafner 6 Mar 06, 2022
Drone detection using YOLOv5

This drone detection system uses YOLOv5 which is a family of object detection architectures and we have trained the model on Drone Dataset. Overview I

Tushar Sarkar 27 Dec 20, 2022
A curated list of resources for Image and Video Deblurring

A curated list of resources for Image and Video Deblurring

Subeesh Vasu 1.7k Jan 01, 2023
Indoor Panorama Planar 3D Reconstruction via Divide and Conquer

HV-plane reconstruction from a single 360 image Code for our paper in CVPR 2021: Indoor Panorama Planar 3D Reconstruction via Divide and Conquer (pape

sunset 36 Jan 03, 2023
InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing

InsTrim The paper: InsTrim: Lightweight Instrumentation for Coverage-guided Fuzzing Build Prerequisite llvm-8.0-dev clang-8.0 cmake = 3.2 Make git cl

75 Dec 23, 2022
.NET bindings for the Pytorch engine

TorchSharp TorchSharp is a .NET library that provides access to the library that powers PyTorch. It is a work in progress, but already provides a .NET

Matteo Interlandi 17 Aug 30, 2021
Manifold-Mixup implementation for fastai V2

Manifold Mixup Unofficial implementation of ManifoldMixup (Proceedings of ICML 19) for fast.ai (V2) based on Shivam Saboo's pytorch implementation of

Nestor Demeure 16 Jul 25, 2022