Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A discord bot to assist you when playing phasmophobia.

phasbot A discord bot to assist you when playing phasmophobia. Add phasbot to your server here! Bot Commands ?help - shows commands ?info [ghost name]

1 Dec 22, 2021
:evergreen_tree: Python module for communicating with the Taiga API

python-taiga A python wrapper for the Taiga REST API. Documentation: https://python-taiga.readthedocs.io/ Usage: : https://python-taiga.readthedocs.io

Nephila 87 Oct 12, 2022
A python script for hitting the kik API to enumerate people based on a username/userlist

kick3d Recon script for enumerating users off of the Kik API. This script has the ability to check single usernames or run through a userlist of usern

Sakura Samurai 19 Oct 04, 2021
AuthGG is a Python library for dealing with Auth.gg apis

AuthGG AuthGG is a Python library for dealing with Auth.gg apis Installation Use the package manager pip to install requests Add the auth.py file in y

ExtremeDev 1 Dec 20, 2021
Python SDK for accessing the Hanko Authentication API

Hanko Authentication SDK for Python This package is maintained by Hanko. Contents Introduction Documentation Installation Usage Prerequisites Create a

Hanko.io 3 Mar 08, 2022
Powerful Telegram bot to countdown to your important events in any group chat.

Powerful Telegram bot to countdown to your important events in any group chat. Live countdown timer.

118 Dec 30, 2022
A simple API Wrapper for Guilded.

Guildr A simple API Wrapper for Guilded. Frequently updated! I am not a user of Guilded, meaning I do not keep track of new Guilded updates or patches

2 Mar 07, 2022
CloudFormation Drift Remediation - Use Cloud Control API to remediate drift that was detected on a CloudFormation stack

CloudFormation Drift Remediation - Use Cloud Control API to remediate drift that was detected on a CloudFormation stack

Cloudar 36 Dec 11, 2022
A simple discord bot based on python

A simple discord bot based on python

SENPAI LEGEND 2 Jul 24, 2022
This is my Discord-Bot named priamoryki-bot based on python.

This is my Discord-Bot named priamoryki-bot based on python. It's a public repository without private information, so you need to correct some code for everything to be working.

priamoryki 2 Dec 14, 2022
Automates the process to obtain an appointment for NIE in spain.

get-nie-appointment A Python script that automates the process of getting an appointment for NIE assignation. It can be modified in order to change th

Ezequiel Aceto 39 Nov 28, 2022
Um bot simples para seguir as pessoas

Um bot simples para seguir pessoas no instagram, criado apeanas para testes. Utilizando o framework "Selenium", criei um bot para entrar em uma conta

Mobben 1 Nov 05, 2021
The source code of the bot that displays erotic images on Discord

説明 このコードはDiscord.pyとNeko APIを使ったNsfw画像表示ボットのソースコードです。 成人向けコンテンツを含むボットなので、不快になる方はこのボットの作成中止をおすすめします。 使い方 まず、install.batを起動してください。 そのあとに、config.json を開き

はなくそ 1 Dec 28, 2021
Códigos pela Força Bruta e Algoritmo Genético para o Problema da Mochila

O problema da mochila é um problema de optimização combinatória. O nome dá-se devido ao modelo de uma situação em que é necessário preencher uma mochi

Hemili Beatriz 1 Jan 08, 2022
A Discord bot that enables using breakout rooms on a server

Discord Breakout Room Bot This bot enables you to use breakout rooms on your Discord server! Note This bot was thrown together within a few hours, so

Till Müller 2 Nov 23, 2021
Mventory is an API-driven solution for Makerspaces, Tinkerers, and Hackers.

Mventory is an API-driven inventory solution for Makers, Makerspaces, Hackspaces, and just about anyone else who needs to keep track of "stuff".

Make Monmouth 107 Dec 21, 2022
A quick-and-dirty script to scrape the daily menu of Leipzig University Mensa and send it to a telegram channel.

Feed me Mensa UL A quick-and-dirty script to scrape the daily menu of Leipzig University Mensa and send it to a telegram channel. For food and cat lov

3 Apr 08, 2022
This Server Cloner can clone the server you want with all the perms of roles in every particular channel.

Server-Cloner-with-perms 🚀 This Server Cloner can clone the server you want with all the perms of roles in every particular channel. Features Clone C

Gripz 0 Feb 17, 2022
The Easy-to-use Dialogue Response Selection Toolkit for Researchers

Easy-to-use toolkit for retrieval-based Chatbot Our released data can be found at this link. Make sure the following steps are adopted to use our code

GMFTBY 32 Nov 13, 2022
An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

URL Uploader Bot An Open Source ALL-In-One Telegram RoBot, that can do lot of things. My Features Installation The Easy Way You can also tap the Deplo

NT BOTS 1 Oct 23, 2021