Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Simple software that can send WhatsApp message to a single or multiple users (including unsaved number**)

wp-automation Info: this is a simple automation software that sends WhatsApp message to single or multiple users. Key feature: -Sends message to multi

3 Jan 31, 2022
a Music bot for discord

Bot this is a discord bot made by AnHalfGuy.py#6031(ID: 747864072879603743) and HastagStopAnimalAbuse#5617(ID :349916852308279306) This Bot Is For Mus

A Discord Bot Development 1 Oct 29, 2021
An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

An Open Source ALL-In-One Telegram RoBot, that can do lot of things.

JOBIN 0 Dec 01, 2021
You can connect with Sanila Ranatunga using this bot😉😉

Sanila-Ranatunga-s-Assistant-Bot You can connect with Sanila Ranatunga using this bot 😉 😉 Reach me on Telegram Sanila's Assistant Bot What is Telegr

Sanila Ranatunga 5 Feb 01, 2022
A Advanced Powerful, Smart And Intelligent Group Management Bot With New And Powerful Features

Vegeta Robot A Advanced Powerful, Smart And Intelligent Group Management Bot With New And Powerful Features ... Written with Pyrogram and Telethon...

⚡ CT_PRO ⚡ 9 Nov 16, 2022
Telegram Group Chat Statistics With Python

Telegram Group Chat Statistics How to Run First add PYTHONPATH in repository root directory enviroment variable by running: export PYTHONPATH=${PWD}

Sina Nazem 3 Apr 18, 2022
Twitter-bot - A Simple Twitter bot with python

twitterbot To use this bot, You will require API Key and Access Key. Signup at h

Bentil Shadrack 8 Nov 18, 2022
Spodcast is a caching Spotify podcast to RSS proxy

Spodcast Spodcast is a caching Spotify podcast to RSS proxy. Using Spodcast you can follow Spotify-hosted netcasts/podcasts using any player which sup

Frank de Lange 260 Jan 01, 2023
Bot made with Microsoft Azure' cloud service

IttenWearBot Autori: Antonio Zizzari Simone Giglio IttenWearBot è un bot intelligente dotato di sofisticate tecniche di machile learning che aiuta gli

Antonio Zizzari 1 Jan 24, 2022
An open-source Discord bot that alerts your server when it's Funky Monkey Friday!

Funky-Monkey-Friday-Bot An open-source Discord bot that alerts your server when it's Funky Monkey Friday! Add it to your server here! https://discord.

Cole Swinford 0 Nov 10, 2022
Telegram bot to stream videos in telegram voicechat for both groups and channels. Supports live strams, YouTube videos and telegram media.

Telegram VCVideoPlayBot An Telegram Bot By @ZauteKm To Stream Videos in Telegram Voice Chat. NOTE: Make sure you have started a VoiceChat in your Grou

Zaute 20 Oct 21, 2022
Análise de dados abertos do programa Taxigov.

Análise de dados do Taxigov Este repositório contém os cadernos Jupyter usados no projeto de análise de dados do Taxigov. Conjunto de dados O conjunto

Augusto Herrmann 1 Jan 10, 2022
Spore REST API asyncio client

Spore REST API asyncio client

LEv145 16 Aug 02, 2022
HackZ-Token-Grabber-V2 - HackZ Token Grabber V2

HackZ-Token-Grabber-V2 was made by Love ❌ code ✅ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ 🌟

! ™NightMare 2 Mar 01, 2022
:globe_with_meridians: A Python wrapper for the Geocodio geolocation service API

Py-Geocodio Python wrapper for Geocodio geocoding API. Full documentation on Read the Docs. If you are upgrading from a version prior to 0.2.0 please

Ben Lopatin 84 Aug 02, 2022
Ditch Xiaomi's cloud and use a Telegram bot instead

Yi-Home_Telegram_Bot_Interface Ditch Xiaomi's cloud and use a Telegram bot instead Features Motion detection Works by monitoring a tmp file that is cr

Erli 10 Aug 18, 2022
A script that takes what you're listening too on Spotify and sets it as your Nertivia custom status.

nertivia-spotify-listening-status A script that takes what you're listening too on Spotify and sets it as your Nertivia custom status. setup Install r

Ben Tettmar 2 Feb 03, 2022
A Terminal User Interface (TUI) for automated trading with Komodo Platform's AtomicDEX-API

PytomicDEX Makerbot A Terminal User Interface (TUI) for automated trading with Komodo Platform's AtomicDEX-API Install sudo apt install wget curl jq g

6 Aug 25, 2022
Autofill HZDR Zeitman entries

Zeitman_autofill Filling out Zeitman is boring. This script might make some of the pain go away. Requirements The selenium package and Chrome webdrive

Tim Callow 8 Mar 14, 2022
just another discord bot

boredbot just another discord bot made to learn python this bots main function is to cache teams meeting links and send them right before the classes

macky 3 Sep 03, 2021