Official repository of IMPROVING DEEP IMAGE MATTING VIA LOCAL SMOOTHNESS ASSUMPTION.

Overview

PWC

IMPROVING DEEP IMAGE MATTING VIA LOCAL SMOOTHNESS ASSUMPTION

This is the official repository of IMPROVING DEEP IMAGE MATTING VIA LOCAL SMOOTHNESS ASSUMPTION. This repo includes all source codes (including data preprocessing code, training code and testing code). Have fun!

Data preparation

We use the training data of Adobe Image Matting to train our model. Please follow the instruction of Adobe Image Matting (AIM) to obtain the training foreground and background as well as the testing data.

Please modify the variable train_path_base in matting/utils/config.py such that the original AIM training foreground images are in the folder train_path_base + "/fg", and place the background images in the folder train_path_base + "/coco_bg", and place the ground truth alpha images in the folder train_path_base + "/alpha".

Please modify the variable test_path_base in matting/utils/config.py to locate the AIM testing data (also called Composition-1k testing data) such that the testing images are in the folder test_path_base + "/merged", and the testing trimaps are in the folder test_path_base + "/trimaps", and the testing ground truth alphas are in the folder test_path_base + "/alpha_copy".

Foreground re-estimation

As described in our paper, the foreground of Adobe Image Matting can be improved to be more consistent with the local smoothness assumption. To obtain the re-estimated foreground by our algorithm, just run python tools/reestimate_foreground_final.py.

Training

To train the model, first click here to download the pretrained encoder model resnetv1d50_b32x8_imagenet_20210531-db14775a.pth from the celebrated repo mmclassification. Place resnetv1d50_b32x8_imagenet_20210531-db14775a.pth in the folder pretrained. Then just run bash train.sh. Without bells and whistles, you will get the state-of-the-art model trained solely on this dataset! By default, the model is trained for the 200 epochs. Note that the reported results in our paper are the models trained for 100 epochs. Thus, you have a great chance to obtain a better model than that reported in our paper!

Testing

In this link, we provide the checkpoint with best performance reported in our paper.

To test our model on the Composition-1k testing data, please place the checkpoint in the folder model. Please change the 105 line of the file matting/models/model.py to for the_step in range(1). This modification in essense disables the backpropagating refinement, or else the testing process costs much time. Then just run bash test.sh.

To test our model on the testing set of AlphaMatting, just place the checkpoint in the folder model and run bash test_alpha_matting.sh.

Acknowledgments

If you use techniques in this project in your research, please cite our paper.

@misc{wang2021ImprovingDeepImageMatting,
      title={Improving Deep Image Matting Via Local Smoothness Assumption}, 
      author={Rui Wang and Jun Xie and Jiacheng Han and Dezhen Qi},
      year={2021},
      eprint={2112.13809},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

If you have any question, please feel free to raise issues!

Below I list some other open source (or partly open source) projects on image matting. I learn a lot from these projects. (For a more comprehensive list of projects on image matting, see wchstrife/Awesome-Image-Matting.) Thank you for sharing your codes! I am proud to be one of you!

Owner
电线杆
电线杆
PyTorch implementation of "MLP-Mixer: An all-MLP Architecture for Vision" Tolstikhin et al. (2021)

mlp-mixer-pytorch PyTorch implementation of "MLP-Mixer: An all-MLP Architecture for Vision" Tolstikhin et al. (2021) Usage import torch from mlp_mixer

isaac 27 Jul 09, 2022
[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo

TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo Lukas Koestler1*    Nan Yang1,2*,†    Niclas Zeller2,3    Daniel Cremers1

TUM Computer Vision Group 744 Jan 04, 2023
Auxiliary Raw Net (ARawNet) is a ASVSpoof detection model taking both raw waveform and handcrafted features as inputs, to balance the trade-off between performance and model complexity.

Overview This repository is an implementation of the Auxiliary Raw Net (ARawNet), which is ASVSpoof detection system taking both raw waveform and hand

6 Jul 08, 2022
Expand human face editing via Global Direction of StyleCLIP, especially to maintain similarity during editing.

Oh-My-Face This project is based on StyleCLIP, RIFE, and encoder4editing, which aims to expand human face editing via Global Direction of StyleCLIP, e

AiLin Huang 51 Nov 17, 2022
Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

Intellia ICT 5 Oct 30, 2022
Repository of best practices for deep learning in Julia, inspired by fastai

FastAI Docs: Stable | Dev FastAI.jl is inspired by fastai, and is a repository of best practices for deep learning in Julia. Its goal is to easily ena

FluxML 532 Jan 02, 2023
A library for Deep Learning Implementations and utils

deeply A Deep Learning library Table of Contents Features Quick Start Usage License Features Python 2.7+ and Python 3.4+ compatible. Quick Start $ pip

Achilles Rasquinha 1 Dec 12, 2022
PyTorch implementation of Densely Connected Time Delay Neural Network

Densely Connected Time Delay Neural Network PyTorch implementation of Densely Connected Time Delay Neural Network (D-TDNN) in our paper "Densely Conne

Ya-Qi Yu 64 Oct 11, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation toolbox based on PyTorch.

traiNNer traiNNer is an open source image and video restoration (super-resolution, denoising, deblurring and others) and image to image translation to

202 Jan 04, 2023
automated systems to assist guarding corona Virus precautions for Closed Rooms (e.g. Halls, offices, etc..)

Automatic-precautionary-guard automated systems to assist guarding corona Virus precautions for Closed Rooms (e.g. Halls, offices, etc..) what is this

badra 0 Jan 06, 2022
Pytorch implementation for "Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter".

Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter This is a pytorch-based implementation for paper Implicit Feature Alignme

wangtianwei 61 Nov 12, 2022
HandFoldingNet ✌️ : A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton

HandFoldingNet ✌️ : A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeleton Wencan Cheng, Jae Hyun Park, Jong

cwc1260 23 Oct 21, 2022
TCPNet - Temporal-attentive-Covariance-Pooling-Networks-for-Video-Recognition

Temporal-attentive-Covariance-Pooling-Networks-for-Video-Recognition This is an implementation of TCPNet. Introduction For video recognition task, a g

Zilin Gao 21 Dec 08, 2022
Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations, CVPR 2019 (Oral)

Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations The code of: Weakly Supervised Learning of Instance Segmentation with I

Jiwoon Ahn 472 Dec 29, 2022
unet-family: Ultimate version

unet-family: Ultimate version 基于之前my-unet代码,我整理出来了这一份终极版本unet-family,方便其他人阅读。 相比于之前的my-unet代码,代码分类更加规范,有条理 对于clone下来的代码不需要修改各种复杂繁琐的路径问题,直接就可以运行。 并且代码有

2 Sep 19, 2022
Tackling data scarcity in Speech Translation using zero-shot multilingual Machine Translation techniques

Tackling data scarcity in Speech Translation using zero-shot multilingual Machine Translation techniques This repository is derived from the NMTGMinor

Tu Anh Dinh 1 Sep 07, 2022
A Multi-modal Model Chinese Spell Checker Released on ACL2021.

ReaLiSe ReaLiSe is a multi-modal Chinese spell checking model. This the office code for the paper Read, Listen, and See: Leveraging Multimodal Informa

DaDa 106 Dec 29, 2022
"Domain Adaptive Semantic Segmentation without Source Data" (ACM MM 2021)

LDBE Pytorch implementation for two papers (the paper will be released soon): "Domain Adaptive Semantic Segmentation without Source Data", ACM MM2021.

benfour 16 Sep 28, 2022