Official Pytorch implementation of 'RoI Tanh-polar Transformer Network for Face Parsing in the Wild.'

Last update: Jan 08, 2023

Related tags

Overview

ibug.face_parsing

RoI Tanh-polar Transformer Network for Face Parsing in the Wild.

Note: If you use this repository in your research, we kindly rquest you to cite the following paper:

@article{lin2021roi,
title = {RoI Tanh-polar transformer network for face parsing in the wild},
journal = {Image and Vision Computing},
volume = {112},
pages = {104190},
year = {2021},
issn = {0262-8856},
doi = {https://doi.org/10.1016/j.imavis.2021.104190},
url = {https://www.sciencedirect.com/science/article/pii/S0262885621000950},
author = {Yiming Lin and Jie Shen and Yujiang Wang and Maja Pantic},
keywords = {Face parsing, In-the-wild dataset, Head pose augmentation, Tanh-polar representation},
}

Dependencies

Numpy: $pip3 install numpy
OpenCV: $pip3 install opencv-python
PyTorch: $pip3 install torch torchvision
ibug.roi_tanh_warping: See this repository for details: https://github.com/ibug-group/roi_tanh_warping.
ibug.face_detection (only needed by the test script): See this repository for details: https://github.com/hhj1897/face_detection.

How to Install

git clone https://github.com/hhj1897/face_parsing
cd face_parsing
git lfs pull
pip install -e .

How to Test

python face_warping_test.py -i 0 -e rtnet50 --decoder fcn -n 11 -d cuda:0

Command-line arguments:

-i VIDEO: Index of the webcam to use (start from 0) or
          path of the input video file
-d: Device to be used by PyTorch (default=cuda:0)
-e: Encoder (default=rtnet50)
--decoder: Decoder (default=fcn)
-n: Number of facial classes, can be 11 or 14 for now (default=11)

iBugMask Dataset

The training and testing images, bounding boxes, landmarks, and parsing maps can be found in the following:

Google Drive

Label Maps

Label map for 11 classes:

0 : background
1 : skin (including face and scalp)
2 : left_eyebrow
3 : right_eyebrow
4 : left_eye
5 : right_eye
6 : nose
7 : upper_lip
8 : inner_mouth
9 : lower_lip
10 : hair

Label map for 14 classes:

0 : background
1 : skin (including face and scalp)
2 : left_eyebrow
3 : right_eyebrow
4 : left_eye
5 : right_eye
6 : nose
7 : upper_lip
8 : inner_mouth
9 : lower_lip
10 : hair
11 : left_ear
12 : right_ear
13 : glasses

Visualisation

You might also like...

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

225 Nov 13, 2022

A python implementation of Yolov5 to detect fire or smoke in the wild in Jetson Xavier nx and Jetson nano

yolov5-fire-smoke-detect-python A python implementation of Yolov5 to detect fire or smoke in the wild in Jetson Xavier nx and Jetson nano You can see

20 Dec 15, 2022

The official implementation of CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing

CSGStumpNet The official implementation of CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing Paper | Project page

39 Dec 26, 2022

Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing

Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing Paper Introduction Multi-task indoor scene understanding is widely considered a

62 Dec 5, 2022

Pytorch implementation of face attention network

Face Attention Network Pytorch implementation of face attention network as described in Face Attention Network: An Effective Face Detector for the Occ

312 Dec 9, 2022

Official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal This is the official pytorch code for SSAT: A Symmetric Semantic-

57 Dec 13, 2022

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Face Identity Disentanglement via Latent Space Mapping - Implement in pytorch with StyleGAN 2 Description Pytorch implementation of the paper Face Ide

58 Dec 24, 2022

Official PyTorch implementation of "Improving Face Recognition with Large AgeGaps by Learning to Distinguish Children" (BMVC 2021)

Inter-Prototype (BMVC 2021): Official Project Webpage This repository provides the official PyTorch implementation of the following paper: Improving F

16 Jun 30, 2022

Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation.

SAFA: Structure Aware Face Animation (3DV2021) Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation. Getting Started

122 Dec 23, 2022

Comments

cannot convert to tflite

Hello, thanks for sharing this great study. I'm researching face parsing and i'm trying to port to Tflite and compare the performance, but I can't since this is using special ops - it uses "grid sample"

What do you suggest I can do in order to test on Tflite/CoreML?

Will training on Lapa dataset improve the accuracy? if not why?

opened by ofirkris 1
_pickle.UnpicklingError: invalid load key, 'v'

How to fix it

Traceback (most recent call last): File "face_parsing_test.py", line 141, in main() File "face_parsing_test.py", line 50, in main face_parser = RTNetPredictor( File "/home/ml/radishevskii/face_parsing/ibug/face_parsing/parser.py", line 81, in init ckpt = torch.load(ckpt, 'cpu') File "/home/ml/radishevskii/anaconda3/envs/inga_vlad/lib/python3.8/site-packages/torch/serialization.py", line 593, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/home/ml/radishevskii/anaconda3/envs/inga_vlad/lib/python3.8/site-packages/torch/serialization.py", line 762, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, 'v'.

opened by vladradishevsky 1
face parsing label

It seems that the dataset released contains only the annotation of 11 facial parts. However, the repository also provide the model trained with dataset containing labels of 14 facial parts. Thus, we wonder how can we get the labels of 14 facial parts. Can you provide the download link? Thanks!

opened by HowToNameMe 0

Releases(v0.2.0)

v0.2.0(Apr 29, 2021)

Source code(tar.gz)
Source code(zip)
v0.1.1(Feb 23, 2021)

Fixed git lfs errors.
Source code(tar.gz)
Source code(zip)

Owner

Jie Shen

GitHub Repository

Code for paper Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting

Decoupled Spatial-Temporal Graph Neural Networks Code for our paper: Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting.

43 Jan 04, 2023

LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference

LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference This repository contains PyTorch evaluation code, training code and pretrained

504 Jan 02, 2023

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.

OpenCOOD OpenCOOD is an Open COOperative Detection framework for autonomous driving. It is also the official implementation of the ICRA 2022 paper OPV

322 Dec 23, 2022

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval, CVPR 2021. Ayan Kumar Bhunia, Pinaki nath Chowdh

22 Aug 27, 2022

Code for the paper: "On the Bottleneck of Graph Neural Networks and Its Practical Implications"

On the Bottleneck of Graph Neural Networks and its Practical Implications This is the official implementation of the paper: On the Bottleneck of Graph

75 Dec 22, 2022

Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)

Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching Official pytorch implementation of "Show, Attend and Distill: Kn

80 Dec 16, 2022

Official Pytorch implementation of 'RoI Tanh-polar Transformer Network for Face Parsing in the Wild.'

Related tags

Overview

ibug.face_parsing

Dependencies

How to Install

How to Test

iBugMask Dataset

Label Maps

Visualisation

You might also like...

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

A python implementation of Yolov5 to detect fire or smoke in the wild in Jetson Xavier nx and Jetson nano

The official implementation of CSG-Stump: A Learning Friendly CSG-Like Representation for Interpretable Shape Parsing

Cerberus Transformer: Joint Semantic, Affordance and Attribute Parsing

Pytorch implementation of face attention network

Official pytorch code for SSAT: A Symmetric Semantic-Aware Transformer Network for Makeup Transfer and Removal

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Official PyTorch implementation of "Improving Face Recognition with Large AgeGaps by Learning to Distinguish Children" (BMVC 2021)

Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation.

Comments

cannot convert to tflite

_pickle.UnpicklingError: invalid load key, 'v'

face parsing label

Releases(v0.2.0)

v0.2.0(Apr 29, 2021)

v0.1.1(Feb 23, 2021)

Owner

Jie Shen

Code for paper Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting

LeViT a Vision Transformer in ConvNet's Clothing for Faster Inference

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

Code for the paper: "On the Bottleneck of Graph Neural Networks and Its Practical Implications"

Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)

Happywhale - Whale and Dolphin Identification Silver🥈 Solution (26/1588)

내가 보려고 정리한 <프로그래밍 기초 Ⅰ> / organized for me

Code for Temporally Abstract Partial Models

CVAT is free, online, interactive video and image annotation tool for computer vision

QR2Pass-project - A proof of concept for an alternative (passwordless) authentication system to a web server

Companion repo of the UCC 2021 paper "Predictive Auto-scaling with OpenStack Monasca"

A PyTorch version of You Only Look at One-level Feature object detector

Face Identity Disentanglement via Latent Space Mapping [SIGGRAPH ASIA 2020]

Code for the paper "Benchmarking and Analyzing Point Cloud Classification under Corruptions"

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

This repository contains the source codes for the paper AtlasNet V2 - Learning Elementary Structures.

Fast Axiomatic Attribution for Neural Networks (NeurIPS*2021)

IDA file loader for UF2, created for the DEFCON 29 hardware badge

A semismooth Newton method for elliptic PDE-constrained optimization