Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Related tags

Deep LearningPPR10K
Overview

Portrait Photo Retouching with PPR10K

Paper | Supplementary Material

PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency
Jie Liang*, Hui Zeng*, Miaomiao Cui, Xuansong Xie and Lei Zhang.
In CVPR 2021.

The proposed Portrait Photo Retouching dataset (PPR10K) is a large-scale and diverse dataset that contains:

  • 11,161 high-quality raw portrait photos (resolutions from 4K to 8K) in 1,681 groups;
  • 3 versions of manual retouched targets of all photos given by 3 expert retouchers;
  • full resolution human-region masks of all photos.

Samples

sample_images

Two example groups of photos from the PPR10K dataset. Top: the raw photos; Bottom: the retouched results from expert-a and the human-region masks. The raw photos exhibit poor visual quality and large variance in subject views, background contexts, lighting conditions and camera settings. In contrast, the retouched results demonstrate both good visual quality (with human-region priority) and group-level consistency.

This dataset is first of its kind to consider the two special and practical requirements of portrait photo retouching task, i.e., Human-Region Priority and Group-Level Consistency. Three main challenges are expected to be tackled in the follow-up researches:

  • Flexible and content-adaptive models for such a diverse task regarding both image contents and lighting conditions;
  • Highly efficient models to process practical resolution from 4K to 8K;
  • Robust and stable models to meet the requirement of group-level consistency.

Agreement

  • All files in the PPR10K dataset are available for non-commercial research purposes only.
  • You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data.

Overview

All data is hosted on GoogleDrive, OneDrive and 百度网盘 (验证码: mrwn):

Path Size Files Format Description
PPR10K-dataset 406 GB 176,072 Main folder
├  raw 313 GB 11,161 RAW All photos in raw format (.CR2, .NEF, .ARW, etc)
├  xmp_source 130 MB 11,161 XMP Default meta-file of the raw photos in CameraRaw, used in our data augmentation
├  xmp_target_a 130 MB 11,161 XMP CameraRaw meta-file of the raw photos recoding the full adjustments by expert a
├  xmp_target_b 130 MB 11,161 XMP CameraRaw meta-file of the raw photos recoding the full adjustments by expert b
├  xmp_target_c 130 MB 11,161 XMP CameraRaw meta-file of the raw photos recoding the full adjustments by expert c
├  masks_full 697 MB 11,161 PNG Full-resolution human-region masks in binary format
├  masks_360p 56 MB 11,161 PNG 360p human-region masks for fast training and validation
├  train_val_images_tif_360p 91 GB 97894 TIF 360p Source (16 bit tiff, with 5 versions of augmented images) and target (8 bit tiff) images for fast training and validation
├  pretrained_models 268 MB 12 PTH pretrained models for all 3 versions
└  hists 624KB 39 PNG Overall statistics of the dataset

One can directly use the 360p (of 540x360 or 360x540 resolution in sRGB color space) training and validation files (photos, 5 versions of augmented photos and the corresponding human-region masks) we have provided following the settings in our paper (train with the first 8,875 files and validate with the last 2286 files).
Also, see the instructions to customize your data (e.g., augment the training samples regarding illuminations and colors, get photos with higher or full resolutions).

Training and Validating the PPR using 3DLUT

Installation

  • Clone this repo.
git clone https://github.com/csjliang/PPR10K
cd PPR10K/code_3DLUT/
  • Install dependencies.
pip install -r requirements.txt
  • Build. Modify the CUDA path in trilinear_cpp/setup.sh adaptively and
cd trilinear_cpp
sh trilinear_cpp/setup.sh

Training

  • Training without HRP and GLC strategy, save models:
python train.py --data_path [path_to_dataset] --gpu_id [gpu_id] --use_mask False --output_dir [path_to_save_models]
  • Training with HRP and without GLC strategy, save models:
python train.py --data_path [path_to_dataset] --gpu_id [gpu_id] --use_mask True --output_dir [path_to_save_models]
  • Training without HRP and with GLC strategy, save models:
python train_GLC.py --data_path [path_to_dataset] --gpu_id [gpu_id] --use_mask False --output_dir [path_to_save_models]
  • Training with both HRP and GLC strategy, save models:
python train_GLC.py --data_path [path_to_dataset] --gpu_id [gpu_id] --use_mask True --output_dir [path_to_save_models]

Evaluation

  • Generate the retouched results:
python validation.py --data_path [path_to_dataset] --gpu_id [gpu_id] --model_dir [path_to_models]
  • Use matlab to calculate the measures in our paper:
calculate_metrics(source_dir, target_dir, mask_dir)

Pretrained Models

mv your/path/to/pretrained_models/* saved_models/
  • specify the --model_dir and --epoch (-1) to validate or initialize the training using the pretrained models, e.g.,
python validation.py --data_path [path_to_dataset] --gpu_id [gpu_id] --model_dir mask_noglc_a --epoch -1
python train.py --data_path [path_to_dataset] --gpu_id [gpu_id] --use_mask True --output_dir mask_noglc_a --epoch -1

Citation

If you use this dataset or code for your research, please cite our paper.

@inproceedings{jie2021PPR10K,
  title={PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency},
  author={Liang, Jie and Zeng, Hui and Cui, Miaomiao and Xie, Xuansong and Zhang, Lei},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2021}
}

Related Projects

3D LUT

Contact

Should you have any questions, please contact me via [email protected].

Easily pull telemetry data and create beautiful visualizations for analysis.

This repository is a work in progress. Anything and everything is subject to change. Porpo Table of Contents Porpo Table of Contents General Informati

Ryan Dawes 33 Nov 30, 2022
The official repository for "Score Transformer: Generating Musical Scores from Note-level Representation" (MMAsia '21)

Score Transformer This is the official repository for "Score Transformer": Score Transformer: Generating Musical Scores from Note-level Representation

22 Dec 22, 2022
Official Pytorch implementation of ICLR 2018 paper Deep Learning for Physical Processes: Integrating Prior Scientific Knowledge.

Deep Learning for Physical Processes: Integrating Prior Scientific Knowledge: Official Pytorch implementation of ICLR 2018 paper Deep Learning for Phy

emmanuel 47 Nov 06, 2022
Official implementation for ICDAR 2021 paper "Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer"

Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer Description Convert offline handwritten mathematical expressi

Wenqi Zhao 87 Dec 27, 2022
Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection

CP-Cluster Confidence Propagation Cluster aims to replace NMS-based methods as a better box fusion framework in 2D/3D Object detection, Instance Segme

Yichun Shen 41 Dec 08, 2022
Using machine learning to predict undergrad college admissions.

College-Prediction Project- Overview: Many have tried, many have failed. Few trailblazers are ambitious enought to chase acceptance into the top 15 un

John H Klinges 1 Jan 05, 2022
Taming Transformers for High-Resolution Image Synthesis

Taming Transformers for High-Resolution Image Synthesis CVPR 2021 (Oral) Taming Transformers for High-Resolution Image Synthesis Patrick Esser*, Robin

CompVis Heidelberg 3.5k Jan 03, 2023
A Pytorch Implementation for Compact Bilinear Pooling.

CompactBilinearPooling-Pytorch A Pytorch Implementation for Compact Bilinear Pooling. Adapted from tensorflow_compact_bilinear_pooling Prerequisites I

169 Dec 23, 2022
Weakly supervised medical named entity classification

Trove Trove is a research framework for building weakly supervised (bio)medical named entity recognition (NER) and other entity attribute classifiers

60 Nov 18, 2022
Public repository containing materials used for Feed Forward (FF) Neural Networks article.

Art041_NN_Feed_Forward Public repository containing materials used for Feed Forward (FF) Neural Networks article. -- Illustration of a very simple Fee

SolClover 2 Dec 29, 2021
To prepare an image processing model to classify the type of disaster based on the image dataset

Disaster Classificiation using CNNs bunnysaini/Disaster-Classificiation Goal To prepare an image processing model to classify the type of disaster bas

Bunny Saini 1 Jan 24, 2022
Analysis of rationale selection in neural rationale models

Neural Rationale Interpretability Analysis We analyze the neural rationale models proposed by Lei et al. (2016) and Bastings et al. (2019), as impleme

Yiming Zheng 3 Aug 31, 2022
Machine learning for NeuroImaging in Python

nilearn Nilearn enables approachable and versatile analyses of brain volumes. It provides statistical and machine-learning tools, with instructive doc

919 Dec 25, 2022
[3DV 2021] Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation This is the official implementation for the method described in Ch

Jiaxing Yan 27 Dec 30, 2022
Python codes for Lite Audio-Visual Speech Enhancement.

Lite Audio-Visual Speech Enhancement (Interspeech 2020) Introduction This is the PyTorch implementation of Lite Audio-Visual Speech Enhancement (LAVSE

Shang-Yi Chuang 85 Dec 01, 2022
Exploring Relational Context for Multi-Task Dense Prediction [ICCV 2021]

Adaptive Task-Relational Context (ATRC) This repository provides source code for the ICCV 2021 paper Exploring Relational Context for Multi-Task Dense

David Brüggemann 35 Dec 05, 2022
Official implementation of "Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks", NeurIPS 2021.

PHDimGeneralization Official implementation of "Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks", NeurIPS 2021. Overvie

Tolga Birdal 13 Nov 08, 2022
This repository implements WGAN_GP.

Image_WGAN_GP This repository implements WGAN_GP. Image_WGAN_GP This repository uses wgan to generate mnist and fashionmnist pictures. Firstly, you ca

Lieon 6 Dec 10, 2021
Multi-resolution SeqMatch based long-term Place Recognition

MRS-SLAM for long-term place recognition In this work, we imply an multi-resolution sambling based visual place recognition method. This work is based

METASLAM 6 Dec 06, 2022
The official code repository for examples in the O'Reilly book 'Generative Deep Learning'

Generative Deep Learning Teaching Machines to paint, write, compose and play The official code repository for examples in the O'Reilly book 'Generativ

David Foster 1.3k Dec 29, 2022