This is the source code for generating the ASL-Skeleton3D and ASL-Phono datasets. Check out the README.md for more details.

Overview

ASL-Skeleton3D and ASL-Phono Datasets Generator

Build Code Quality DOI - ASL-Skeleton3D DOI - ASL-Phono

The ASL-Skeleton3D contains a representation based on mapping into the three-dimensional space the coordinates of the signers in the ASLLVD dataset. The ASL-Phono, in turn, introduces a novel linguistics-based representation, which describes the signs in the ASLLVD dataset in terms of a set of attributes of the American Sign Language phonology.

This is the source code used to generate the ASL-Skeleton3D and ASL-Phono datasets, which are based on the American Sign Language Lexicon Video Dataset (ASLLVD).

Learn more about the datasets:

  • Paper: "ASL-Skeleton3D and ASL-Phono: Two NovelDatasets for the American Sign Language" -> CIn

Download

Download the processed datasets by using the links below:

Generate

If you prefer generating the datasets by yourself, this section presents the requirements, setup and procedures to execute the code.

The generation is a process comprising the phases below, which start by the retrieval of the original ASLLVD samples for then computing additional properties, as follows:

  • download: original samples (video sequences) are obtained from the ASLLVD.
  • segment: signs are segmented from the original samples.
  • skeleton: signer skeletons are estimated.
  • normalize: the coordinates of the skeletons are normalized.
  • phonology: the phonological attributes are extracted.

Requirements

To generate the datasets, your system will need the following software configured:

OpenPose will require additional hardware and software configured which might include a NVIDIA GPU and related drivers and software. Please, check this link for the full list.

Recommended

If you prefer running a Docker container with the software requirements already configured, check out the link below -- just make sure to have a GPU available to your Docker environment:

Installation

Once observed the requirements, checkout the source code and execute the following command, which will setup your virtual environment and dependencies:

$ poetry install

Configuration

There is a set of files in the folder ./config that will help you to configure the parameters for generating the datasets. A good starting point is to take a look into the ./config/template.yaml file, which contains a basic structure with all the properties documented.

You will also find other predefined configurations that might help you to generate the datasets. Just remember to always review the comments inside of the files to fine-tune the execution to your environment.

Learn about the configurations available in the ./config/template.yaml, which contains the properties documented.

Generation

ASL-Skeleton3D

The ASL-Skeleton3D is generated by using the configuration predefined in the file ./config/asl-skeleton3d.yaml. Thus, to start processing the dataset, execute the following command informing this file as the parameter -c (or --config):

$ poetry run python main.py -c ./config/asl-skeleton3d.yaml

The resulting dataset will be located in the folder configured as output for the phase normalize, which by default is set to ../work/dataset/normalized.

ASL-Phono

The ASL-Skeleton3D is generated by using the configuration predefined in the file ./config/asl-phono.yaml. Thus, to start processing the dataset, execute the following command informing this file as the parameter -c (or --config):

$ poetry run python main.py -c ./config/asl-phono.yaml

The resulting dataset will be located in the folder configured as output for the phase phonology, which by default is set to ../work/dataset/phonology.

Logs

The logs from the datasets processing will be recorded in the file ./output.log.

Deprecated datasets

Previously, we introduced the dataset ASLLVD-Skeleton, which is now being replaced by the ASL-Skeleton3D. Read more about the old dataset in the links:

Citation

Please cite the following paper if you use this repository in your reseach.

@article{asl-datasets-2021,
  title     = {ASL-Skeleton3D and ASL-Phono: Two Novel Datasets for the American Sign Language},
  author    = {Cleison Correia de Amorim and Cleber Zanchettin},
  year      = {2021},
}

Contact

For any question, feel free to contact me at:

You might also like...
Official implementation of the paper 'Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution' in CVPR 2022
Official implementation of the paper 'Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution' in CVPR 2022

LDL Paper | Supplementary Material Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution Jie Liang*, Hu

Source code for
Source code for "MusCaps: Generating Captions for Music Audio" (IJCNN 2021)

MusCaps: Generating Captions for Music Audio Ilaria Manco1 2, Emmanouil Benetos1, Elio Quinton2, Gyorgy Fazekas1 1 Queen Mary University of London, 2

The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.
The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding.

SuperGen The source code for Generating Training Data with Language Models: Towards Zero-Shot Language Understanding. Requirements Before running, you

Source code, datasets and trained models for the paper Learning Advanced Mathematical Computations from Examples (ICLR 2021), by François Charton, Amaury Hayat (ENPC-Rutgers) and Guillaume Lample

Maths from examples - Learning advanced mathematical computations from examples This is the source code and data sets relevant to the paper Learning a

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Official code for Next Check-ins Prediction via History and Friendship on Location-Based Social Networks (MDM 2018)
Official code for Next Check-ins Prediction via History and Friendship on Location-Based Social Networks (MDM 2018)

MUC Next Check-ins Prediction via History and Friendship on Location-Based Social Networks (MDM 2018) Performance Details for Accuracy: | Dataset

The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

ood-text-emnlp Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them" Files fine_tune.py is used to finetune the GPT-2 mo

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Comments
  • keypoint scale?

    keypoint scale?

    Hello this data looks to be amazing, but making use of it takes a bit more knowledge about how to actually translate the x,y values into usable points.

    It seems you guys have taken advantage of the --keypoint_scale in OpenPose - could you post something about how to translate these decimal numbers back into something more like a traditional x,y value? I'd like to draw these points using standard javascript, but right now I can't figure how how to rescale them back to size.

    Any help would be greatly appreciated!

    opened by mspanish 0
Releases(v1.0.0)
Owner
Cleison Amorim
Cleison Amorim
An adaptive hierarchical energy management strategy for hybrid electric vehicles

An adaptive hierarchical energy management strategy This project contains the source code of an adaptive hierarchical EMS combining heuristic equivale

19 Dec 13, 2022
Official code for Next Check-ins Prediction via History and Friendship on Location-Based Social Networks (MDM 2018)

MUC Next Check-ins Prediction via History and Friendship on Location-Based Social Networks (MDM 2018) Performance Details for Accuracy: | Dataset

Yijun Su 3 Oct 09, 2022
Unofficial implementation of Fast-SCNN: Fast Semantic Segmentation Network

Fast-SCNN: Fast Semantic Segmentation Network Unofficial implementation of the model architecture of Fast-SCNN. Real-time Semantic Segmentation and mo

Philip Popien 69 Aug 11, 2022
机器学习、深度学习、自然语言处理等人工智能基础知识总结。

说明 机器学习、深度学习、自然语言处理基础知识总结。 目前主要参考李航老师的《统计学习方法》一书,也有一些内容例如XGBoost、聚类、深度学习相关内容、NLP相关内容等是书中未提及的。

Peter 445 Dec 12, 2022
Implementation for the EMNLP 2021 paper "Interactive Machine Comprehension with Dynamic Knowledge Graphs".

Interactive Machine Comprehension with Dynamic Knowledge Graphs Implementation for the EMNLP 2021 paper. Dependencies apt-get -y update apt-get instal

Xingdi (Eric) Yuan 19 Aug 23, 2022
一套完整的微博舆情分析流程代码,包括微博爬虫、LDA主题分析和情感分析。

已经将项目的关键文件上传,包含微博爬虫、LDA主题分析和情感分析三个部分。 1.微博爬虫 实现微博评论爬取和微博用户信息爬取,一天大概十万条。 2.LDA主题分析 实现文档主题抽取,包括数据清洗及分词、主题数的确定(主题一致性和困惑度)和最优主题模型的选择(暴力搜索)。 3.情感分析 实现评论文本的

182 Jan 02, 2023
Code repository for Semantic Terrain Classification for Off-Road Autonomous Driving

BEVNet Datasets Datasets should be put inside data/. For example, data/semantic_kitti_4class_100x100. Training BEVNet-S Example: cd experiments bash t

(Brian) JoonHo Lee 24 Dec 12, 2022
The hippynn python package - a modular library for atomistic machine learning with pytorch.

The hippynn python package - a modular library for atomistic machine learning with pytorch. We aim to provide a powerful library for the training of a

Los Alamos National Laboratory 37 Dec 29, 2022
Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator

DRL-robot-navigation Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator. Using Twin Delayed Deep Deterministic Policy Gra

87 Jan 07, 2023
A big endian Gentoo port developed on a Pine64.org RockPro64

Gentoo-aarch64_be A big endian Gentoo port developed on a Pine64.org RockPro64 The endian wars are over... little endian won. As a result, it is incre

Rory Bolt 6 Dec 07, 2022
Speech Recognition using DeepSpeech2.

deepspeech.pytorch Implementation of DeepSpeech2 for PyTorch using PyTorch Lightning. The repo supports training/testing and inference using the DeepS

Sean Naren 2k Jan 04, 2023
PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems

PowerGridworld provides users with a lightweight, modular, and customizable framework for creating power-systems-focused, multi-agent Gym environments that readily integrate with existing training fr

National Renewable Energy Laboratory 37 Dec 17, 2022
Repository of Vision Transformer with Deformable Attention

Vision Transformer with Deformable Attention This repository contains the code for the paper Vision Transformer with Deformable Attention [arXiv]. Int

410 Jan 03, 2023
Differentiable Factor Graph Optimization for Learning Smoothers @ IROS 2021

Differentiable Factor Graph Optimization for Learning Smoothers Overview Status Setup Datasets Training Evaluation Acknowledgements Overview Code rele

Brent Yi 60 Nov 14, 2022
Face Alignment using python

Face Alignment Face Alignment using python Input Image Aligned Face Aligned Face Aligned Face Input Image Aligned Face Input Image Aligned Face Instal

Sajjad Aemmi 28 Nov 23, 2022
Multi-Joint dynamics with Contact. A general purpose physics simulator.

MuJoCo Physics MuJoCo stands for Multi-Joint dynamics with Contact. It is a general purpose physics engine that aims to facilitate research and develo

DeepMind 5.2k Jan 02, 2023
Virtual hand gesture mouse using a webcam

NonMouse 日本語のREADMEはこちら This is an application that allows you to use your hand itself as a mouse. The program uses a web camera to recognize your han

Yuki Takeyama 55 Jan 01, 2023
SegNet-like Autoencoders in TensorFlow

SegNet SegNet is a TensorFlow implementation of the segmentation network proposed by Kendall et al., with cool features like strided deconvolution, a

Andrea Azzini 66 Nov 05, 2021
Memory-efficient optimum einsum using opt_einsum planning and PyTorch kernels.

opt-einsum-torch There have been many implementations of Einstein's summation. numpy's numpy.einsum is the least efficient one as it only runs in sing

Haoyan Huo 9 Nov 18, 2022
A treasure chest for visual recognition powered by PaddlePaddle

简体中文 | English PaddleClas 简介 飞桨图像识别套件PaddleClas是飞桨为工业界和学术界所准备的一个图像识别任务的工具集,助力使用者训练出更好的视觉模型和应用落地。 近期更新 2021.11.1 发布PP-ShiTu技术报告,新增饮料识别demo 2021.10.23 发

4.6k Dec 31, 2022