Source code for CVPR 2021 paper "Riggable 3D Face Reconstruction via In-Network Optimization"

Related tags

Deep LearningINORig
Overview

Riggable 3D Face Reconstruction via In-Network Optimization

Source code for CVPR 2021 paper "Riggable 3D Face Reconstruction via In-Network Optimization".

[paper] [supp] [arXiv] [presen_video] [supp_video].

Installation

(1) Create an Anaconda environment.

conda env create -f env.yaml
conda activate INORig

(2) Clone the repository and install dependencies.

git clone https://github.com/zqbai-jeremy/INORig.git
cd INORig
pip install -r requirements_pip.txt

(3) Setup 3DMM

mkdir external
cd external
git clone https://github.com/zqbai-jeremy/face3d.git
cd face3d
  • Setup face3d as in YadiraF/face3d.

  • Download "Exp_Pca.bin" from Guo et al. (in "CoarseData" link of their repository) and copy to "<INORig directory>/external/face3d/examples/Data/BFM/Out/".

  • Download "std_exp.txt" from Deng et al. and copy to "<INORig directory>/external/face3d/examples/Data/BFM/Out/".

(5) Download pre-trained model (Due to the sensitivity of face swapping, please email [email protected] to request for the models. Please use your institution email and indicate in the email that you agree to only use the models for research purpose and not to share with 3rd parties. Sorry for the inconvenience and thank you for your understanding!) to "<INORig directory>/net_weights/". Need to create the folder. Unzip to get .pth files. "Ours.pth" is the basic version. "Ours(R).pth" is a more robust while less accurate version. Experiments in the paper are performed with these models.

Run Demo

  • Modify directory paths in demo.py and run
cd <INORig_directory>
python demo.py
  • Variables:

    • rig_img_dir: Folder contains images to build the rig.

    • src_vid_path: Video path to drive the rig.

    • out_dir: Folder for outputs. Reconstructions of images are in "<out_dir>/mesh" and "<out_dir>/visualization". Video reconstruction and retargeting are in "<out_dir>/videos".

  • The example video clip "<INORig directory>/examples/videos/clip1.mp4" is from https://www.youtube.com/watch?v=ikAfrKf5A8I

Acknowledge

  • Thank Ayush Tewari, Soubhik Sanyal and Jiaxiang Shang for helping the evaluations!

  • Thank for the helpful tools from YadiraF/face3d!

Citation

@InProceedings{Bai_2021_CVPR,
    author    = {Bai, Ziqian and Cui, Zhaopeng and Liu, Xiaoming and Tan, Ping},
    title     = {Riggable 3D Face Reconstruction via In-Network Optimization},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {6216-6225}
}
Pneumonia Detection using machine learning - with PyTorch

Pneumonia Detection Pneumonia Detection using machine learning. Training was done in colab: DEMO: Result (Confusion Matrix): Data I uploaded my datase

Wilhelm Berghammer 12 Jul 07, 2022
Deep functional residue identification

DeepFRI Deep functional residue identification Citing @article {Gligorijevic2019, author = {Gligorijevic, Vladimir and Renfrew, P. Douglas and Koscio

Flatiron Institute 156 Dec 25, 2022
Neural network-based build time estimation for additive manufacturing

Neural network-based build time estimation for additive manufacturing Oh, Y., Sharp, M., Sprock, T., & Kwon, S. (2021). Neural network-based build tim

Yosep 1 Nov 15, 2021
Face-Recognition-based-Attendance-System - An implementation of Attendance System in python.

Face-Recognition-based-Attendance-System A real time implementation of Attendance System in python. Pre-requisites To understand the implentation of F

Muhammad Zain Ul Haque 1 Dec 31, 2021
Domain Generalization for Mammography Detection via Multi-style and Multi-view Contrastive Learning

MSVCL_MICCAI2021 Installation Please follow the instruction in pytorch-CycleGAN-and-pix2pix to install. Example Usage An example of vendor-styles tran

Jaron Lee 11 Oct 19, 2022
PyTorch implementation of Self-supervised Contrastive Regularization for DG (SelfReg)

SelfReg PyTorch official implementation of Self-supervised Contrastive Regularization for Domain Generalization (SelfReg, https://arxiv.org/abs/2104.0

64 Dec 16, 2022
Show-attend-and-tell - TensorFlow Implementation of "Show, Attend and Tell"

Show, Attend and Tell Update (December 2, 2016) TensorFlow implementation of Show, Attend and Tell: Neural Image Caption Generation with Visual Attent

Yunjey Choi 902 Nov 29, 2022
Implementation of CVPR 2020 Dual Super-Resolution Learning for Semantic Segmentation

Dual super-resolution learning for semantic segmentation 2021-01-02 Subpixel Update Happy new year! The 2020-12-29 update of SISR with subpixel conv p

Sam 79 Nov 24, 2022
PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

Yuzhi ZHAO 11 Jul 28, 2022
Simulation of the solar system using various nummerical methods

solar-system Simulation of the solar system using various nummerical methods Download the repo Make shure matplotlib, scipy etc. are installed execute

Caspar 7 Jul 15, 2022
Python scripts form performing stereo depth estimation using the CoEx model in ONNX.

ONNX-CoEx-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the CoEx model in ONNX. Stereo depth estimation on the

Ibai Gorordo 8 Dec 29, 2022
A fast implementation of bss_eval metrics for blind source separation

fast_bss_eval Do you have a zillion BSS audio files to process and it is taking days ? Is your simulation never ending ? Fear no more! fast_bss_eval i

Robin Scheibler 99 Dec 13, 2022
Dashboard for the COVID19 spread

COVID-19 Data Explorer App A streamlit Dashboard for the COVID-19 spread. The app is live at: [https://covid19.cwerner.ai]. New data is queried from G

Christian Werner 22 Sep 29, 2022
Predicts an answer in yes or no.

Oui-ou-non-prediction Predicts an answer in 'yes' or 'no'. It is based on the game 'effeuiller la marguerite' in which the person plucks flower petals

Ananya Gupta 1 Jan 15, 2022
[CVPR 2021 Oral] Variational Relational Point Completion Network

VRCNet: Variational Relational Point Completion Network This repository contains the PyTorch implementation of the paper: Variational Relational Point

PL 121 Dec 12, 2022
Neural Network Libraries

Neural Network Libraries Neural Network Libraries is a deep learning framework that is intended to be used for research, development and production. W

Sony 2.6k Dec 30, 2022
Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention

cosFormer Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention Update log 2022/2/28 Add core code License This

120 Dec 15, 2022
Multi-Anchor Active Domain Adaptation for Semantic Segmentation (ICCV 2021 Oral)

Multi-Anchor Active Domain Adaptation for Semantic Segmentation Munan Ning*, Donghuan Lu*, Dong Wei†, Cheng Bian, Chenglang Yuan, Shuang Yu, Kai Ma, Y

Munan Ning 36 Dec 07, 2022
PyTorch code for our paper "Gated Multiple Feedback Network for Image Super-Resolution" (BMVC2019)

Gated Multiple Feedback Network for Image Super-Resolution This repository contains the PyTorch implementation for the proposed GMFN [arXiv]. The fram

Qilei Li 66 Nov 03, 2022
Learning Time-Critical Responses for Interactive Character Control

Learning Time-Critical Responses for Interactive Character Control Abstract This code implements the paper Learning Time-Critical Responses for Intera

Movement Research Lab 227 Dec 31, 2022