Human Action Controller - A human action controller running on different platforms.

Last update: Jul 20, 2022

Related tags

Overview

Human Action Controller (HAC)

Goal

A human action controller running on different platforms.

Fun	Easy-to-use
Accurate	Anywhere

Fun Examples

Mouse Control

Keyboard Control

Playing Game

Enhancing interaction

Solutions provided by HAC

Platform	Module	Progress
PC / Win10	Mouse Control	V
PC / Win10	Keyboard Control	V
PC / Ubuntu	Mouse Control
PC / Ubuntu	Keyboard Control

Getting started

Installation

$ pip install pyhac

Run the demo of mouse control

$ git clone https://github.com/dabit-lucas/hac.git
$ cd hac
$ python demo.py

Recording custom actions

$ python recording.py -d {action name} -k True

Press key "r" to start recording, the data will be saved into ./data

Training a custom module

Here is an example of a config file of action set,

{
    "actions": [
        "r_five",
        "r_zero",
        "l_five",
        "l_zero",
        "two_index_fingers_up",
        "two_index_fingers_down",
        "33",
        "55",
        "sit"
    ],
    "type": "gesture_only"
}

These actions form a model by running a training process:

$ python train.py --conf {path_of_action} --model_name {name_of_model}

The generated model will become a module. Take mouse control module as an exmaple, it can create mappings among actions and controls by the following code:

mouse_module = hac.add_module("mouse_control")
hac.set_init_module(mouse_module)

# create mapping between controls and actions
mouse_module.add_mouse_mapping("mouse_left_down", ["r_five", "r_zero"])
mouse_module.add_mouse_mapping("mouse_left_up", "r_five")
mouse_module.add_mouse_mapping("mouse_right_down", ["l_five", "l_zero"])
mouse_module.add_mouse_mapping("mouse_right_up", "l_five")
mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])
mouse_module.add_mouse_mapping("right_move_diff", ["r_zero", "r_zero"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_five", "l_five"])
mouse_module.add_mouse_mapping("left_move_diff", ["l_zero", "l_zero"])
mouse_module.add_mouse_mapping("roll_up", "two_index_fingers_up")
mouse_module.add_mouse_mapping("roll_down", "two_index_fingers_down")

If the five gesture with a right hand shows in consecutive two frames ["r_five", "r_five"], then do control right_move_diff, which means moving the mouse cursor. The above description can be represented by the following code:

mouse_module.add_mouse_mapping("right_move_diff", ["r_five", "r_five"])

Development guideline

The structure of HAC

Community

Welcome to ask any question in issues.

Contributing

Any contribution is welcomed. Please fork this repo and summit a pull request.

Human Action Controller - A human action controller running on different platforms.

Related tags

Overview

Human Action Controller (HAC)

Goal

Fun Examples

Mouse Control

Keyboard Control

Playing Game

Enhancing interaction

Solutions provided by HAC

Getting started

Installation

Run the demo of mouse control

Recording custom actions

Training a custom module

Development guideline

Community

Contributing

Owner

Source codes for Improved Few-Shot Visual Classification (CVPR 2020), Enhancing Few-Shot Image Classification with Unlabelled Examples

Implementation of CVAE. Trained CVAE on faces from UTKFace Dataset to produce synthetic faces with a given degree of happiness/smileyness.

Learning nonlinear operators via DeepONet

Code reproduce for paper "Vehicle Re-identification with Viewpoint-aware Metric Learning"

Solution to the Weather4cast 2021 challenge

Social Fabric: Tubelet Compositions for Video Relation Detection

OpenMMLab Pose Estimation Toolbox and Benchmark.

Tensorflow 2 implementation of the paper: Learning and Evaluating Representations for Deep One-class Classification published at ICLR 2021

Code & Models for Temporal Segment Networks (TSN) in ECCV 2016

"Neural Turing Machine" in Tensorflow

The repository for freeCodeCamp's YouTube course, Algorithmic Trading in Python

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Understanding the Effects of Datasets Characteristics on Offline Reinforcement Learning

Code to reproduce the experiments in the paper "Transformer Based Multi-Source Domain Adaptation" (EMNLP 2020)

Sound Event Detection with FilterAugment

MPViT:Multi-Path Vision Transformer for Dense Prediction

ADB-IP-ROTATION - Use your mobile phone to gain a temporary IP address using ADB and data tethering

A simple code to perform canny edge contrast detection on images.

FS2KToolbox FS2K Dataset Towards the translation between Face

Code for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in Deep Latent Space"