Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
A Pancakeswap v2 trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more.

Pancakeswap v2 trading client A Pancakeswap trading client (and bot) with limit orders, stop-loss, custom gas strategies, a GUI and much more. If you

571 Mar 15, 2022
Набор утилит для Discord с использованием языка программирования Python.

Discord Tools v0.1 Functions: WebHook spamer Spotify account generator (What?) QR Code Token stealer Token generator Discord nitro gen/check Discor to

Максим Скризов 3 Aug 23, 2022
Template to create a telegram bot in python

Template for Telegram Bot Template to create a telegram bot in python. How to Run First add src to PYTHONPATH: export PYTHONPATH=${PWD} Then run: pyt

Ali Hejazizo 12 Dec 24, 2022
A Discord bot that generates inspirational quotes & motivating messages whenever a user is sad

Encourage bot is a discord bot that allows users to randomly get Inspirational quotes messages and gives motivational encouragements whenever someone says that he's sad/depressed.

1 Nov 25, 2021
Utilizing the freqtrade high-frequency cryptocurrency trading framework to build and optimize trading strategies. The bot runs nonstop on a Rasberry Pi.

Freqtrade Strategy Repository Please test all scripts and dry run them before using them in live mode Contact me on discord if you have any questions!

Michael Fourie 90 Jan 01, 2023
A bot that updates about the most subscribed artist' channels on YouTube

A bot that updates about the most subscribed artist' channels on YouTube. A weekly top chart report is provided every Monday. It posts updates on Twitter

Marco Fantauzzo 5 Dec 14, 2022
GUI Pancakeswap V2 and Uniswap V3 trading client (and bot)MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC

GUI Pancakeswap 2 and Uniswap 3 trading client (and bot) (MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC) UPDATE: MUTI TRADE TOKEN ENABLE ,TRADE 1

2 Dec 27, 2021
Python wrapper for the Sportradar APIs ⚽️🏈

Sportradar APIs This is a Python wrapper for the sports APIs provided by Sportradar. You'll need to sign up for an API key to use the service. Sportra

John W. Miller 39 Jan 01, 2023
Discord-Token-Formatter - A simple script to convert discord tokens from email token to token only format

Discord-Token-Formatter A simple script to convert discord tokens from email:pas

2 Oct 23, 2022
A small repository with convenience functions for working with the Notion API.

Welcome! Within this respository are a few convenience functions to assist with the pulling and pushing of data from the Notion API.

10 Jul 09, 2022
Download archived malware from ActiveState's source code mirror

malware-archivist (ma) Tool to aid security researchers in dissecting malware. Often, repository maintainers will remove malicious packages entirely f

ActiveState Software 28 Dec 12, 2022
Georeferencing large amounts of data for free.

Geolocate Georeferencing large amounts of data for free. Special thanks to @brunodepauloalmeida and the whole team for the contributions. How? It's us

Gabriel Gazola Milan 23 Dec 30, 2022
A simple script that loads and hot-reloads cogs when you save any changes

DiscordBot-HotReload A simple script that loads and hot-reloads cogs when you save any changes Usage @bot.event async def on_ready(): from HotRelo

2 Jan 14, 2022
My homeserver setup. Everything managed securely using Portainer.

homeserver-traefik-portainer Features: access all services with free TLS from letsencrypt using your own domain running a side project is super simple

Tomasz Wójcik 44 Jan 03, 2023
A python client for the Software-Challenge Germany.

sc-client-python A python client for the Software-Challenge Germany. Creating a new project (Optional) Install virtualenv virtualenv is a tool that cr

rpkak 3 Jan 22, 2022
Azure DevOps Extension for Azure CLI

Azure DevOps Extension for Azure CLI The Azure DevOps Extension for Azure CLI adds Pipelines, Boards, Repos, Artifacts and DevOps commands to the Azur

1 Nov 03, 2021
Balanced API library in python.

Balanced Online Marketplace Payments v1.x requires Balanced API 1.1. Use v0.x for Balanced API 1.0. Installation pip install balanced Usage View Bala

Balanced 70 Oct 04, 2022
Diablo II Resurrected helper

Diablo II Resurrected 快捷施法辅助 功能: + 创建守护进程,注册全局热键 alt+/ 启用和关闭功能 (todo: 播放声音提示) + 按 x 强制移动 + 按 1 ~ 0 快捷施法到鼠标区域 使用 编辑配置 settings.py 技能信息做如下定义: SKILLS:

Wan 2 Nov 06, 2022
Bot Auto Chess.com

Bot Auto Chess.com Is a suggestion for chess moves on the chess.com platform. The available features are: chess suggestions and moves automatically. i

Tn. Ninja 34 Jan 01, 2023
A user reconnaisance tool that extracts a target's information from Instagram, DockerHub & Github.

A user reconnaisance tool that extracts a target's information from Instagram, DockerHub & Github. Also searches for matching usernames on Github.

Richard Mwewa 127 Dec 22, 2022