A warping based image translation model focusing on upper body synthesis.

Overview

Pose2Img

Upper body image synthesis from skeleton(Keypoints). Sub module in the ICCV-2021 paper "Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates". [arxiv / github]

This is a modified implementation of Synthesizing Images of Humans in Unseen Poses.

Setup

To install dependencies, run

pip install -r requirements.txt

To run this module, you need two NVIDIA gpus with at least 11 GB respectively. Our code is tested on Ubuntu 18.04LTS with Python3.6.

Train on the custom dataset

For your own dataset, you need to modify custom config.yaml.

python main.py \
    --name test \
    --config_path {config_path}.yaml \
    --batch_size 1 \
  • The raw keypoints for each frame is of shape (3, 137)

Demo

Generate a realistic video for Oliver from {keypoints}.npz. You need to do some minor modifications to $configs/yaml/oliver.yaml$.

python inference.py \
    --cfg_path configs/yaml/oliver.yaml \
    --name test \
    --npz_path target_pose/oliver_pose.npz \
    --wav_path target_pose/oliver_audio.wav
  • In the result directory, you can find jpg files which correspond to the npz.

Demo Dataset and Checkpoint

  • Coming soon

Citation

If you find this code useful for your research, please use the following BibTeX entry.

@inproceedings{qian2021speech,
  title={Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates},
  author={Qian, Shenhan and Tu, Zhi and Zhi, YiHao and Liu, Wen and Gao, Shenghua},
  journal={International Conference on Computer Vision (ICCV)},
  year={2021}
}
Owner
zhiyh
zhiyh
An esoteric visual language that takes image files as input based on a multi-tape turing machine, designed for compatibility with C.

vizh An esoteric visual language that takes image files as input based on a multi-tape turing machine, designed for compatibility with C. Overview Her

Sy Brand 228 Dec 17, 2022
An automated Comic Book downloader (cbr/cbz) for use with SABnzbd, NZBGet and torrents

Mylar Note that feature development has stopped as we have moved to Mylar3. EOL for this project is the end of 2020 and will no longer be supported. T

979 Dec 13, 2022
Simple utility to tinker with OPlus images

OPlus image utilities Prerequisites Linux running kernel 5.4 or up (check with uname -r) Image rebuilding Used to rebuild read-only erofs images into

Wiley Lau 15 Dec 28, 2022
将位图转为彩色矢量 svg 图片

一个将位图描摹为彩色矢量 svg 图片的程序,是一个命令行工具,使用 Python 脚本实现,运行环境 Python3.8+。 ✨ 效果 以一个字帖图片为例,这是 png 格式的位图(370KB): 这是颜

Haujet Zhao 104 Dec 30, 2022
Python binding to Skia Graphics Library

Skia python binding Python binding to Skia Graphics Library. Binding based on pybind11. Currently, the binding is under active development. Install Bi

Kota Yamaguchi 170 Jan 06, 2023
Converting Images Into Minecraft Houses

Converting Images Into Minecraft Houses In this particular project, we turned a 2D Image into Minecraft pixel art and then scaled it in 3D such that i

Mathias Oliver Valdbjørn Jørgensen 1 Feb 02, 2022
⚡ZenGL is a minimalist Python module providing exactly one way to render scenes with OpenGL.

ZenGL is a minimalist Python module providing exactly one way to render scenes with OpenGL.

Szabolcs Dombi 133 Dec 17, 2022
HtmlWebShot - A python3 package which Can Create Images From url, Html-CSS, Svg and from any readable file and texts with many setup features.

A python3 package which Can Create Images From url, Html-CSS, Svg and from any readable file and texts with many setup features

Danish 24 Dec 14, 2022
Python library that finds the size / type of an image given its URI by fetching as little as needed

FastImage This is an implementation of the excellent Ruby library FastImage - but for Python. FastImage finds the size or type of an image given its u

Brian Muller 28 Mar 01, 2022
Create a QR-code Generator app using only Python.

QR-code_Generator Create a QR-code Generator app using only Python. This apps generated a QR code for a single link. Libraryes used in this app -- py

Soham P Phasalkar 1 Oct 17, 2021
Sample data for the napari image viewer.

napari-demo-data Sample data for the napari image viewer. This napari plugin was generated with Cookiecutter using @napari's cookiecutter-napari-plugi

Genevieve Buckley 1 Nov 08, 2021
Python-based tools for document analysis and OCR

ocropy OCRopus is a collection of document analysis programs, not a turn-key OCR system. In order to apply it to your documents, you may need to do so

OCRopus 3.2k Jan 04, 2023
API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language

You can try it @ api-fatoora api-fatoora API to help generating QR-code for ZATCA's e-invoice known as Fatoora with any programming language Disclaime

نافع الهلالي 12 Oct 05, 2022
Labelme is a graphical image annotation tool, It is written in Python and uses Qt for its graphical interface

Image Polygonal Annotation with Python (polygon, rectangle, circle, line, point and image-level flag annotation).

Kentaro Wada 9.6k Jan 09, 2023
PIX is an image processing library in JAX, for JAX.

PIX PIX is an image processing library in JAX, for JAX. Overview JAX is a library resulting from the union of Autograd and XLA for high-performance ma

DeepMind 294 Jan 08, 2023
Photini - A free, easy to use, digital photograph metadata (Exif, IPTC, XMP) editing application for Linux, Windows and MacOS.

A free, easy to use, digital photograph metadata (Exif, IPTC, XMP) editing application for Linux, Windows and MacOS. "Metadata" is said to mea

Jim Easterbrook 120 Dec 20, 2022
reversable image censoring tool

StupidCensor a REVERSABLE image censoring tool to reversably mosiac censor jpeg files to temporarily remove image details not allowed on most websites

2 Jan 28, 2022
Python pygame project that turns your images to matrix rain

Matrix-Rain-An-Image This project implements the classic Matrix digital rain effect in python with pygame to build up an image provided with multiple

7 Dec 11, 2022
Utilities for SteamVR on Linux

This project contains scripts to improve the functionally of SteamVR on Linux:

86 Dec 29, 2022
Maze generator with most popular shapes - hexagon, triangle, square

Maze-Generator Maze generator with most popular shapes - hexagon, triangle, square (sqaure not implemented yet): Theory: Planar Graph https://en.wikip

Kacper Plesiak 2 Dec 28, 2021