Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Telegram Bot to store Posts and Documents and it can Access by Special Links.

File-sharing-Bot Telegram Bot to store Posts and Documents and it can Access by Special Links. I Guess This Will Be Usefull For Many People..... 😇 .

Code X Botz 1.2k Jan 08, 2023
A Python library for loading data from a SpaceX Starlink satellite.

Starlink Python A Python library for loading data from a SpaceX Starlink satellite. The goal is to be a simple interface for Starlink. It builds upon

Austin 2 Jan 16, 2022
Discord Token Generator - Python (Generates Tokens and Joins your Server Automatically) hCaptcha Bypass **FREE**

Best Discord Token Generator {hCaptcha bypass FREE Unlimited Memberboost} Install few requirements & run main.py it will redirect you to the Download

1 Oct 27, 2021
Secure Tunnel Manager

Making life easy of those who are in need of OpenSource alternative of AWS Secure Tunnel.

Suyash Chavan 1 Sep 27, 2022
A python bot that stops muck chains

muck-chains-stopper-bot a bot that stops muck chains this is the source code of u/DaniDevChainBreaker (the main r/DaniDev muck chains breaker) guys th

24 Jan 04, 2023
Please Do Not Throw Sausage Pizza Away - Side Scrolling Up The OSI Stack

Please Do Not Throw Sausage Pizza Away - Side Scrolling Up The OSI Stack

John Capobianco 2 Jan 25, 2022
An Async Bot/API wrapper for Twitch made in Python.

TwitchIO is an asynchronous Python wrapper around the Twitch API and IRC, with a powerful command extension for creating Twitch Chat Bots. TwitchIO co

TwitchIO 590 Jan 03, 2023
A supabase client for python

supabase-client A Supabase client for Python. This mirrors the design of supabase-js Full documentation: https://keosariel.github.io/2021/08/08/supaba

kenneth gabriel 11 Dec 19, 2022
TORNADO CASH Pancakeswap Sniper BOT 2022-V1 (MAC WINDOWS ANDROID LINUX)

TORNADO CASH Pancakeswap Sniper BOT 2022-V1 (MAC WINDOWS ANDROID LINUX)

Crypto Trader 1 Jan 06, 2022
A project that alerts me when there's a dog outside so I can go look at it.

Dog Detector A project that alerts me when there's a dog outside so I can go look at it. Tech Specs This script uses the YOLOv3 object detection model

rydercalmdown 58 Jul 29, 2022
OMDB-and-TasteDive-Mashup - Mashing up data from two different APIs to make movie recommendations.

OMDB-and-TasteDive-Mashup This hadns-on project is in the Python 3 Programming Specialization offered by University of Michigan via Coursera. Mashing

Eszter Pai 1 Jan 05, 2022
Generate discord nitro codes and check them

Discord Nitro Generator and Checker A discord nitro generator and checker for all your nitro needs Explore the docs » Report Bug · Request Feature · J

509 Jan 02, 2023
Simple Discord bot for snekbox (sandboxed Python code execution), self-host or use a global instance

snakeboxed Simple Discord bot for snekbox (sandboxed Python code execution), self-host or use a global instance

0 Jun 25, 2022
LHXP・Official "LH - Cyber Security" Discord Leveling-Bot

LHXP・Official "LH - Cyber Security" Discord Leveling-Bot Based on nsde/NOVΛLIX Feature Overview /clear @user Requires admin permission Purges all XP

Felix・onlix 2 Mar 09, 2022
A Anything goes Discord bot written in python and uses the wrapper Discord.py

GerardTheWizard A Anything goes Discord bot written in python and uses the wrapper Discord.py What can he do? Allow users to level up through typing,

1 May 05, 2022
A simple MTProto-based bot that can download various types of media (>10MB) on a local storage

TG Media Downloader Bot 🤖 A telegram bot based on Pyrogram that downloads on a local storage the following media files: animation, audio, document, p

Alessio Tudisco 11 Nov 01, 2022
Modified Version of mega.py package for Pyrogram Bots

Pyro Mega.py Python library for the Mega.co.nz API, currently supporting: login uploading downloading deleting searching sharing renaming moving files

I'm Not A Bot #Left_TG 10 Aug 03, 2022
A free, minimal, lightweight, cross-platform, easily expandable Twitch IRC/API bot.

parky's twitch bot A free, minimal, lightweight, cross-platform, easily expandable Twitch IRC/API bot. Features 🔌 Connect to Twitch IRC chat! 🔌 Conn

Andreas Schneider 10 Dec 30, 2022
Visualização de dados do TaxiGov.

Visualização de dados do TaxiGov Este repositório apresenta uma visualização das corridas de táxi do programa TaxiGov do governo federal, realizadas n

Ministério da Economia do Brasil 5 Dec 20, 2022