Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Unfollows Users You're Following

Github-Unfollow-Bot Info It unfollows users you're following, it runs in the background so you can still do what you do without it bothering you. It's

ExT 4 Sep 03, 2022
Telegram Reporter

[Telegram Reporter v.3 ] 🇮🇷 AliCybeRR 🇮🇷 [ AliCybeRR.Reporter feature ] Login Your Telegram account 👽 support Termux ❕ No Limits ⚡ Secure 🔐 Free

AliCybeRR 1 Jun 08, 2022
Github action for automatically determine the version for next release by using repository tags

This action will automatically determine the version for next release by using repository tags

Igor Gov 7 Oct 25, 2022
Shows VRML team stats of all players in your pubs

VRML Team Stat Searcher Displays Team Name, Team Rank (Worldwide), and tier of all the players in your pubs. GUI WIP: Username search works & pub name

Hamish 2 Dec 22, 2022
SpamBot.py allows you, to spam other Chat Partners etc.

SpamBot -SpamBot.py allows you, to spam other Chat Partners etc. Install If you downloaded it yet, you have to install "requirements.txt" write the di

Marco 1 Jan 16, 2022
S3-cleaner - A Python script attempts to delete the all objects/delete markers/versions from specific S3 bucket

Remove All Objects From S3 Bucket This Python script attempts to delete the all

9 Jan 27, 2022
Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

SUBIN 449 Dec 27, 2022
Auslesen, entschlüsseln und parsen von Smart Meter Telegrammen

Netz-NÖ SmartMeter-P1-Reader https://www.netz-noe.at/Download-(1)/Smart-Meter/218_9_SmartMeter_Kundenschnittstelle_lektoriert_14.aspx Dieses Skript en

3 Jan 14, 2022
A Google Charts API for Python, meant to be used as an alternative to matplotlib.

GooPyCharts A Google Charts API for Python 2 and 3, meant to be used as an alternative to matplotlib. Syntax is similar to MATLAB. The goal of this pr

Sagnik Ghosh 202 Oct 04, 2022
Desktop Backup Client for Borg

Vorta Backup Client Vorta is a backup client for macOS and Linux desktops. It integrates the mighty BorgBackup with your desktop environment to protec

BorgBase.com 1.5k Jan 03, 2023
A twitter multi-tool for OSINT on twitter accounts.

TwitterCheckr A twitter multi-tool for OSINT on twitter accounts. Infomation TwitterCheckr also known as TCheckr is multi-tool for OSINT on twitter a

IRIS 16 Dec 23, 2022
A very tiny python api for the stock exchange tradegate.de

pytradegate A very tiny python api for the stock exchange tradegate.de The api provides the recent ask/bid data and all other data as found on the det

dunderstr aka seimen 7 Aug 24, 2022
Asynchronous Python API Wrapper for phisherman.gg

Asynchronous Python API Wrapper for phisherman.gg

Qrista Labs 4 Apr 30, 2022
Simple Bot With Python 3.8+ For Converstaion Your Media

Media-Conversation Simple Bot With Python 3.8+ For Converstaion Your Media

Farzin 2 Dec 06, 2021
DadBot 2.0 is an interactive bot that reflects the personality of a typical dad

DadBot 2.0 is an interactive bot that reflects the personality of a typical dad! You can start by running main while all directories are maintained as they are on this GitHub.

1 Dec 04, 2021
Script que envia e-mails de denúncia para desativar número de WhatsApp.

SpamReport (Alpha) Este script foi feito apenas para uso educacional, não me responsabilizo por qualquer uso indevido. Version: 1.0 Alpha Ative essa o

Kiny-Kiny 83 Dec 20, 2022
Instagram boosting

instagram boosting bot This bot can boost your instagram account! Rules and Instruction Use git clone to download this repository Open cmd/terminal an

Eskimo 4 Oct 20, 2022
The modern Lavalink wrapper designed for discord.py

Pomice The modern Lavalink wrapper designed for discord.py This library is heavily based off of/uses code from the following libraries: Wavelink Slate

Gstone 1 Feb 02, 2022
A simple weather information tool.

pwy A simple weather information tool. Table of Contents Features Dependencies Installation Usage Update Changelog Credits License Features ASCII art

Clint 105 Dec 31, 2022
Telegram Link Wayback Bot. This bot archives a web page thrown at itself with wayback Machine (Archive.org).

Telegram Link Wayback Bot. This bot archives a web page thrown at itself with wayback Machine (Archive.org).

Hüzünlü Artemis [HuzunluArtemis] 11 Feb 18, 2022