Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild (CVPR 2022)

Overview

HandLer

This repository contains the code and data for the following paper:

Forward Propagation, Backward Regression, and Pose Association for Hand Tracking in the Wild (CVPR 2022).

We develop a method to detect and track multiple hands in videos.

Installation

Install Torch 1.7.1

pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html

Install other requirements

pip install -r requirements.txt

Install Detectron2 (version 0.4)

For CUDA 11.0 and Torch 1.7.x

python -m pip install detectron2==0.4 -f \
  https://dl.fbaipublicfiles.com/detectron2/wheels/cu110/torch1.7/index.html

Youtube-Hand Dataset

Please find the Youtube-Hand dataset in the project page.

Go to main folder

cd ./projects/HandLer

Model Weights

Please find the pretraind model weights here:

https://drive.google.com/file/d/1eaZn9E7lHvXY3Fh90f2eIK1P2yZwNfnP/view?usp=sharing

Please find the weights trained on Youtube-Hand training set here:

https://drive.google.com/file/d/1sK5bRTHt5zLOVPruUt-zyZ_AOrBvGQ9-/view?usp=sharing

Hand Tracking Demo: Run HandLer on custom videos

python demo.py --cfg configs/yt_trk.yaml --output output_vid.mp4 --weights /path/to/weights --input input_vid.mp4

Training on Youtube-Hand Dataset

For training on single GPU:

python train_trk.py --config-file configs/yt_trk.yaml

For training on multiple GPUs:

python train_trk.py --config-file configs/yt_trk.yaml --num_gpu 4

Testing on Youtube-Hand Test set

python tracking.py --cfg /path/to/cfg --out_dir /path/to/results/folders --weights /path/to/weights --root_path /path/to/dataset

Evaluating on Youtube-Hand Test set

python evaluations.py /path/to/results/folders

Improving Hand Tracking using Hand-Body Association

By associating hands with human bodies and linking human bodies across frames, we can establish correspondence between detected instances of the same hand across different frames, reducing identity switches in tracking. To know more about this, please check our work Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild

Citation

If you find our code or dataset useful, please cite:

@inproceedings{handler_2022,
      title={Forward Propagation, Backward Regression and Pose Association for Hand Tracking in the Wild},
      author={Mingzhen Huang and Supreeth Narasimhaswamy and Saif Vazir and Haibin Ling and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }

@inproceedings{bodyhands_2022,
      title={Whose Hands Are These? Hand Detection and Hand-Body Association in the Wild},
      author={Supreeth Narasimhaswamy and Thanh Nguyen and Mingzhen Huang and Minh Hoai},
      booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2022},
  }
Owner
[email protected]
Computer Vision Lab at Stony Brook University
<a href=[email protected]">
Calendars for various securities exchanges.

IMPORTANT NOTE This package is currently unmaintained as the sponsor, quantopian, is going through corporate changes. As such there is a fork of this

Quantopian, Inc. 545 Jan 07, 2023
Easy Discord Webhook Token Grabber!

Easy Discord Webhook Token Grabber!

†† 27 Jun 01, 2022
You can submit any PR and have SWAGS. Happy Hacktoberfest !

Excluded project Repository 🔴 🔴 🔴 - PR limit is reached. Please use another Repository Hacktoberfest 2021 🎉 🗣 Hacktoberfest encourages participat

Hansajith 63 Oct 21, 2022
Ma2tl - macOS forensic timeline generator using the analysis result DBs of mac apt

ma2tl (mac_apt to timeline) This is a DFIR tool for generating a macOS forensic

Minoru Kobayashi 66 Nov 18, 2022
A program to convert YouTube channel registration information into Json files for ThirdTube.

ThirdTubeImporter A program to convert YouTube channel registration information into Json files for ThirdTube. Usage Japanese https://takeout.google.c

Hidegon 2 Dec 18, 2021
A telegram smoot and high quality music player bot.

▪︎ Music Player ▪︎ A smooth telegram music bot with high quality songs ■ [Features] • Fast Starts streaming your inputs while downloading and converti

Simple Boy 3 Feb 05, 2022
A python API for BSCScan (Binance Smart Chain Explorer), available on PyPI.

bscscan-python A complete Python API for BscScan.com, available on PyPI. Powered by BscScan.com APIs. This is a gently modified fork of the etherscan-

Panagiotis Kotsias 246 Dec 31, 2022
Bifrost C2. Open-source post-exploitation using Discord API

Bifrost Command and Control What's Bifrost? Bifrost is an open-source Discord BOT that works as Command and Control (C2). This C2 uses Discord API for

38 Dec 05, 2022
Cord Python API Client

Cord Python API Client The data programming platform for AI 💻 Features Minimal low-level Python client that allows you to interact with Cord's API Su

Cord 52 Nov 25, 2022
Bomber-X - A SMS Bomber made with Python

Bomber-X A SMS Bomber made with Python Linux/Termux apt update apt upgrade apt i

S M Shahriar Zarir 2 Mar 10, 2022
ZenML 🙏: MLOps framework to create reproducible ML pipelines for production machine learning.

ZenML is an extensible, open-source MLOps framework to create production-ready machine learning pipelines. It has a simple, flexible syntax, is cloud and tool agnostic, and has interfaces/abstraction

ZenML 2.6k Dec 27, 2022
EZXT - A ccxt wrapped client for binance & ftx

EZXT Open source & beginner-friendly ccxt wrapped client for binance & ftx Want

Shaft 10 Oct 25, 2022
Yuichixspam - TLEEGRAM SPAM BOT For Python

𝒀𝑼𝑰𝑪𝑯𝑰 ✘ 𝑺𝑷𝑨𝑴 𝑩𝑶𝑻ノ 🚀 Deploy on Heroku (https://heroku.com/deploy?t

MOHIT X PANDIT 6 Jan 30, 2022
Debugging with Stack Overflow: Web Search Behavior in Novice and Expert Programmers

Debugging with Stack Overflow: ICSE SEET, 2022 This is the GitHub repository associated with the 2022 ICSE SEET paper, Debugging with Stack Overflow:

Madeline Endres 1 Jan 31, 2022
A Discord bot to scrape textfiles from messages and put them to Hastebin

A Discord bot to scrape textfiles from messages and put them to Hastebin. Intended to use on support servers to help users read textfiles on mobile.

1 Jan 23, 2022
A self-hosted Discord music bot.

Cassette A self-hosted Discord music bot. Requirements py-cord pynacl pytube Setup Intended to be hosted on Heroku. Fork or clone this repo. Create a

Lohan 8 Apr 28, 2022
A Python API For Questionnaire

Инструкция по разворачиванию приложения Окружение проекта: python 3.8 Django 2.2.10 djangorestframework Склонируйте репозиторий с помощью git: git clo

2 Feb 14, 2022
A Discord BOT that uses Google Sheets for storing the roles and permissions of a discord server.

Discord Role Manager Bot Role Manager is a discord BOT that utilizes Google Sheets for the organization of a server's hierarchy and permissions. Detai

Dion Rigatos 17 Oct 13, 2022
Morpheus is a telegram bot that helps to simplify the process of making custom telegram stickers.

😎 Morpheus is a telegram bot that helps to simplify the process of making custom telegram stickers. As you may know, Telegram's official Sti

Abhijith K S 1 Dec 14, 2022
A Python Script to automate searching of available vaccination centers in the city and hence booking

Cowin Vaccine Availability Notifier Cowin Vaccine Availability Notifier takes your City or PIN code as an input and automatically notifies you via ema

Jayesh Padhiar 7 Sep 05, 2021