A collection of resources on GAN Inversion.

Overview

awesome gan-inversion papers

Awesome Maintenance PR's Welcome

This repo is a collection of resources on GAN inversion, as a supplement for our survey:

@article{xia2021survey,
  author    = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan},
  title     = {GAN Inversion: A Survey},
  journal={arXiv preprint arXiv: 2101.05278},
  year={2021}
}

Contributing

Feedback and contributions are welcome!

If you think I have missed out on something (or) have any suggestions (papers, implementations and other resources), feel free to pull a request.

I have released the latex files. Please pull a request, open an issue, or send me an email if you find any inappropriate expressions of the survey.

markdown format:

**Here is the Paper Name.**
*[Author 1](homepage), Author 2, and Author 3.*
Conference or Journal Year. [[PDF](link)] [[Project](link)] [[Github](link)] [[Video](link)] [[Data](link)]

Survey

[Papers on Generative Modeling]

GAN Inversion: A Survey.
Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, Ming-Hsuan Yang.
arxiv 2021. [PDF]

inverted pretrained model

StyleGAN2-Ada: Training Generative Adversarial Networks with Limited Data.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila.
NeurIPS 2020. [PDF] [Github] [Steam StyleGAN2-ADA]

StyleGAN2: Analyzing and Improving the Image Quality of StyleGAN.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila.
CVPR 2020. [PDF] [Offical TF] [PyTorch] [Unoffical Tensorflow 2.0]

StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks.
Tero Karras, Samuli Laine, Timo Aila.
CVPR 2019. [PDF] [Offical TF]

ProGAN: Progressive Growing of GANs for Improved Quality, Stability, and Variation.
Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen.
ICLR 2018. [PDF] [Offical TF]

inversion method

This part contatins generatal inversion methods, while methods in the next application part are mainly designed for specific tasks.

Using Latent Space Regression to Analyze and Leverage Compositionality in GANs.
Lucy Chai, Jonas Wulff, Phillip Isola.
ICLR 2021. [PDF] [Github] [Project] [Colab]

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs.
Hui-Po Wang, Ning Yu, Mario Fritz.
CVPR 2021. [PDF]

e4e: Designing an Encoder for StyleGAN Image Manipulation.
Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, Daniel Cohen-Or.
arxiv 2021. [PDF] [Github]

Enjoy Your Editing: Controllable GANs for Image Editing via Latent Space Navigation.
Peiye Zhuang, Oluwasanmi Koyejo, Alexander G. Schwing.
ICLR 2021. [PDF]

Improved StyleGAN Embedding: Where are the Good Latents?
Peihao Zhu, Rameen Abdal, Yipeng Qin, Peter Wonka.
arxiv 2020. [PDF]

Learning a Deep Reinforcement Learning Policy Over the Latent Space of a Pre-trained GAN for Semantic Age Manipulation.
Kumar Shubham, Gopalakrishnan Venkatesh, Reijul Sachdev, Akshi, Dinesh Babu Jayagopi, G. Srinivasaraghavan.
arxiv 2020. [PDF]

Lifting 2D StyleGAN for 3D-Aware Face Generation.
Yichun Shi, Divyansh Aggarwal, Anil K. Jain.
arxiv 2020. [PDF]

Navigating the GAN Parameter Space for Semantic Image Editing.
Anton Cherepkov, Andrey Voynov, Artem Babenko.
arxiv 2020. [PDF] [Github]

Augmentation-Interpolative AutoEncoders for Unsupervised Few-Shot Image Generation.
Davis Wertheimer, Omid Poursaeed, Bharath Hariharan.
arxiv 2020. [PDF]

Mask-Guided Discovery of Semantic Manifolds in Generative Models.
Mengyu Yang, David Rokeby, Xavier Snelgrove.
Workshop on Machine Learning for Creativity and Design (NeurIPS) 2020. [PDF] [Github]

Unsupervised Discovery of Disentangled Manifolds in GANs.
Yu-Ding Lu, Hsin-Ying Lee, Hung-Yu Tseng, Ming-Hsuan Yang.
arxiv 2020. [PDF]]

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation.
Zongze Wu, Dani Lischinski, Eli Shechtman.
arxiv 2020. [PDF]

GAN Steerability without optimization.
Nurit Spingarn-Eliezer, Ron Banner, Tomer Michaeli.
ICLR 2021. [OpenReview] [PDF]

On The Inversion Of Deep Generative Models (When and How Can Deep Generative Models be Inverted?).
Aviad Aberdam, Dror Simon, Michael Elad.
arxiv 2020. [PDF] [OpenReview]

PIE: Portrait Image Embedding for Semantic Control.
A. Tewari, M. Elgharib, M. BR, F. Bernard, H-P. Seidel, P. P‌érez, M. Zollhöfer, C.Theobalt.
SIGGRAPH Asia 2020. [PDF] [Project]

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation.
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, Daniel Cohen-Or.
CVPR 2021. [PDF] [Github] [Project]

GAN-Control: Explicitly Controllable GANs.
Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, Gerard Medioni.
arxiv 2021. [PDF]

Understanding the Role of Individual Units in a Deep Neural Network.
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, Antonio Torralba.
National Academy of Sciences 2020. [PDF] [Github] [Project]

GHFeat: Generative Hierarchical Features from Synthesizing Images.
Yinghao Xu, Yujun Shen, Jiapeng Zhu, Ceyuan Yang, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

SeFa: Closed-Form Factorization of Latent Semantics in GANs.
Yujun Shen, Bolei Zhou.
CVPR 2021. [PDF] [Github] [Project]

Collaborative Learning for Faster StyleGAN Embedding.
Shanyan Guan, Ying Tai, Bingbing Ni, Feida Zhu, Feiyue Huang, Xiaokang Yang.
arxiv 2020. [PDF]

Disentangling in Latent Space by Harnessing a Pretrained Generator.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
arxiv 2020. [PDF]

Face Identity Disentanglement via Latent Space Mapping.
Yotam Nitzan, Amit Bermano, Yangyan Li, Daniel Cohen-Or.
SIGGRAPH Asia (TOG) 2020. [PDF] [Github]

Transforming and Projecting Images into Class-conditional Generative Networks.
Minyoung Huh, Richard Zhang, Jun-Yan Zhu, Sylvain Paris, Aaron Hertzmann.
ECCV 2020. [PDF] [Github] [Project]

Interpreting the Latent Space of GANs via Correlation Analysis for Controllable Concept Manipulation.
Ziqiang Li, Rentuo Tao, Hongjing Niu, Bin Li.
arxiv 2020. [PDF]

Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space.
Jonas Wulff, Antonio Torralba.
arxiv 2020. [PDF]

GANSpace: Discovering Interpretable GAN Controls.
Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, Sylvain Paris.
NeurIPS 2020. [PDF] [Github]

MimicGAN: Robust Projection onto Image Manifolds with Corruption Mimicking.
Rushil Anirudh, Jayaraman J. Thiagarajan, Bhavya Kailkhura, Timo Bremer.
IJCV 2020. [PDF]

StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows.
Rameen Abdal, Peihao Zhu, Niloy Mitra, Peter Wonka.
Siggraph (TOG) 2021. [PDF] [Github]

Rewriting a Deep Generative Model.
David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba.
ECCV 2020. [PDF] [Github]

StyleGAN2 Distillation for Feed-forward Image Manipulation.
Yuri Viazovetskyi, Vladimir Ivashkin, Evgeny Kashin.
ECCV 2020. [PDF] [Github]

In-Domain GAN Inversion for Real Image Editing.
Jiapeng Zhu, Yujun Shen, Deli Zhao, Bolei Zhou.
ECCV 2020. [PDF] [Project] [Github]

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation.
Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo.
ECCV 2020. [PDF] [Github]

On the "steerability" of generative adversarial networks.
Ali Jahanian, Lucy Chai, Phillip Isola.
ICLR 2020. [PDF] [Project]

Unsupervised Discovery of Interpretable Directions in the GAN Latent Space.
Andrey Voynov, Artem Babenko.
ICML 2020. [PDF] [Github]

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models.
Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis.
CVPR 2020. [PDF]

A Disentangling Invertible Interpretation Network for Explaining Latent Representations.
Patrick Esser, Robin Rombach, Björn Ommer.
CVPR 2020. [PDF] [Project] [Github]

Editing in Style: Uncovering the Local Semantics of GANs.
Edo Collins, Raja Bala, Bob Price, Sabine Süsstrunk.
CVPR 2020. [PDF] [Github]

Image Processing Using Multi-Code GAN Prior.
Jinjin Gu, Yujun Shen, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Interpreting the Latent Space of GANs for Semantic Face Editing.
Yujun Shen, Jinjin Gu, Xiaoou Tang, Bolei Zhou.
CVPR 2020. [PDF] [Project] [Github]

Image2StyleGAN++: How to Edit the Embedded Images?
Rameen Abdal, Yipeng Qin, Peter Wonka.
CVPR 2020. [PDF]

Semantic Photo Manipulation with a Generative Image Prior.
David Bau, Hendrik Strobelt, William Peebles, Jonas, Bolei Zhou, Jun-Yan Zhu, Antonio Torralba.
SIGGRAPH 2019. [PDF]

Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?
Rameen Abdal, Yipeng Qin, Peter Wonka.
ICCV 2019. [PDF] [Github]

Seeing What a GAN Cannot Generate.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF] [PDF]

GAN-based Projector for Faster Recovery with Convergence Guarantees in Linear Inverse Problems.
Ankit Raj, Yuqi Li, Yoram Bresler.
ICCV 2019. [PDF]

Inverting Layers of a Large Generator.
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, Antonio Torralba.
ICCV 2019. [PDF]

Inverting The Generator Of A Generative Adversarial Network (II).
Antonia Creswell, Anil A Bharath.
TNNLS 2018. [PDF] [Github]

Invertibility of Convolutional Generative Networks from Partial Measurements.
Fangchang Ma, Ulas Ayaz, Sertac Karaman.
NeurIPS 2018. [PDF] [Github]

Metrics for Deep Generative Models.
Nutan Chen, Alexej Klushyn, Richard Kurle, Xueyan Jiang, Justin Bayer, Patrick van der Smagt.
AISTATS 2018. [PDF]

Towards Understanding the Invertibility of Convolutional Neural Networks.
Anna C. Gilbert, Yi Zhang, Kibok Lee, Yuting Zhang, Honglak Lee.
IJCAI 2017. [PDF]

One Network to Solve Them All - Solving Linear Inverse Problems using Deep Projection Models.
J. H. Rick Chang, Chun-Liang Li, Barnabas Poczos, B. V. K. Vijaya Kumar, Aswin C. Sankaranarayanan.
ICCV 2017. [PDF]

Precise Recovery of Latent Vectors from Generative Adversarial Networks.
Zachary C. Lipton, Subarna Tripathi.
ICLR 2017 workshop. [PDF] [Github]

Inverting The Generator Of A Generative Adversarial Network.
Antonia Creswell, Anil Anthony Bharath.
NIPSW 2016. [PDF]

Generative Visual Manipulation on the Natural Image Manifold.
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, Alexei A. Efros.
ECCV 2016. [PDF]

application

content generation

Paint by Word.
David Bau, Alex Andonian, Audrey Cui, YeonHwan Park, Ali Jahanian, Aude Oliva, Antonio Torralba.
arxiv 2021. [PDF]

Unsupervised Image Transformation Learning via Generative Adversarial Networks.
Kaiwen Zha, Yujun Shen, Bolei Zhou.
arxiv 2021. [PDF] [Project]

TediGAN: Text-Guided Diverse Image Generation and Manipulation.
Weihao Xia, Yujiu Yang, Jing-Hao Xue, Baoyuan Wu.
CVPR 2021. [PDF] [Data] [Github]

LOHO: Latent Optimization of Hairstyles via Orthogonalization.
Rohit Saha, Brendan Duke, Florian Shkurti, Graham W. Taylor, Parham Aarabi.
CVPR 2021. [PDF] [Github]

SAM: Only a Matter of Style-Age Transformation Using a Style-Based Regression Model.
Yuval Alaluf, Or Patashnik, Daniel Cohen-Or.
arxiv 2021. [PDF] [Github]

OSTeC: One-Shot Texture Completion.
Baris Gecer, Jiankang Deng, Stefanos Zafeiriou.
arxiv 2021. [PDF] [Github]

GAN2Shape: Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs.
Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo.
ICLR 2021 (oral). [PDF] [Github] [Project]

Exploring Adversarial Fake Images on Face Manifold.
Dongze Li, Wei Wang, Hongxing Fan, Jing Dong.
arxiv 2021. [PDF]

Generating Images from Caption and Vice Versa via CLIP-Guided Generative Latent Space Search.
Federico A. Galatolo, Mario G.C.A. Cimino, Gigliola Vaglini.
arxiv 2021. [PDF]

Unsupervised Image-to-Image Translation via Pre-trained StyleGAN2 Network.
Jialu Huang, Jing Liao, Sam Kwong.
arxiv 2020. [PDF]

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs.
yaxing wang, Lu Yu, Joost van de Weijer.
NeurIPS 2020. [PDF] [Github]

DeepLandscape: Adversarial Modeling of Landscape Videos.
E. Logacheva, R. Suvorov, O. Khomenko, A. Mashikhin, and V. Lempitsky.
ECCV 2020. [PDF] [Github] [Project]

image restoration

GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution.
Kelvin C.K. Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, Chen Change Loy.
CVPR 2021. [PDF] [Project] [Github]

GFP-GAN: Towards Real-World Blind Face Restoration with Generative Facial Prior.
Xintao Wang, Yu Li, Honglun Zhang, Ying Shan.
arxiv 2021. [PDF] [Project]

image understanding

Repurposing GANs for One-shot Semantic Part Segmentation.
Nontawat Tritrong, Pitchaporn Rewatbowornwong, Supasorn Suwajanakorn.
CVPR 2021 (oral). [PDF] [Project] [Github]

compressed sensing

Generator Surgery for Compressed Sensing.
Niklas Smedemark-Margulies, Jung Yeon Park, Max Daniels, Rose Yu, Jan-Willem van de Meent, Paul Hand.
arxiv 2021. [PDF] [Github]

Task-Aware Compressed Sensing with Generative Adversarial Networks.
Maya Kabkab, Pouya Samangouei, Rama Chellappa.
AAAI 2018. [PDF]

acknowledgement

Thanks for the feedback from Jun-Yan Zhu, Andrey Voynov, and Rushil Anirudh.

PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)

2021-CVPR-MvCLN This repo contains the code and data of the following paper accepted by CVPR 2021 Partially View-aligned Representation Learning with

XLearning Group 33 Nov 01, 2022
Codes for "Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier"

Deep-RTC [project page] This repository contains the source code accompanying our ECCV 2020 paper. Solving Long-tailed Recognition with Deep Realistic

Gina Wu 16 May 26, 2022
PyTorch implementation of "Simple and Deep Graph Convolutional Networks"

Simple and Deep Graph Convolutional Networks This repository contains a PyTorch implementation of "Simple and Deep Graph Convolutional Networks".(http

chenm 253 Dec 08, 2022
Using deep learning model to detect breast cancer.

Breast-Cancer-Detection Breast cancer is the most frequent cancer among women, with around one in every 19 women at risk. The number of cases of breas

1 Feb 13, 2022
Parris, the automated infrastructure setup tool for machine learning algorithms.

README Parris, the automated infrastructure setup tool for machine learning algorithms. What Is This Tool? Parris is a tool for automating the trainin

Joseph Greene 319 Aug 02, 2022
Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN)

Multi-Stage Spatial-Temporal Convolutional Neural Network (MS-GCN) This code implements the skeleton-based action segmentation MS-GCN model from Autom

Benjamin Filtjens 8 Nov 29, 2022
Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

VariationalRecurrentNeuralNetwork Pytorch implementation of the Variational RNN (VRNN), from A Recurrent Latent Variable Model for Sequential Data. Th

emmanuel 251 Dec 17, 2022
Detecting Potentially Harmful and Protective Suicide-related Content on Twitter

TwitterSuicideML Scripts for reproducing the Machine Learning analysis of the paper: Detecting Potentially Harmful and Protective Suicide-related Cont

3 Oct 17, 2022
2nd solution of ICDAR 2021 Competition on Scientific Literature Parsing, Task B.

TableMASTER-mmocr Contents About The Project Method Description Dependency Getting Started Prerequisites Installation Usage Data preprocess Train Infe

Jianquan Ye 298 Dec 21, 2022
Official PyTorch Implementation of HELP: Hardware-adaptive Efficient Latency Prediction for NAS via Meta-Learning (NeurIPS 2021 Spotlight)

[NeurIPS 2021 Spotlight] HELP: Hardware-adaptive Efficient Latency Prediction for NAS via Meta-Learning [Paper] This is Official PyTorch implementatio

42 Nov 01, 2022
Code & Data for the Paper "Time Masking for Temporal Language Models", WSDM 2022

Time Masking for Temporal Language Models This repository provides a reference implementation of the paper: Time Masking for Temporal Language Models

Guy Rosin 12 Jan 06, 2023
Dyalog-apl-docset - Dyalog APL Dash Docset Generator

Dyalog APL Dash Docset Generator o alasa e kili sona kepeken tenpo lili a A Dash

Maciej Goszczycki 1 Jan 10, 2022
The official repo of the CVPR 2021 paper Group Collaborative Learning for Co-Salient Object Detection .

GCoNet The official repo of the CVPR 2021 paper Group Collaborative Learning for Co-Salient Object Detection . Trained model Download final_gconet.pth

Qi Fan 46 Nov 17, 2022
Codebase for Diffusion Models Beat GANS on Image Synthesis.

Codebase for Diffusion Models Beat GANS on Image Synthesis.

Katherine Crowson 128 Dec 02, 2022
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in Tensorflow Lite.

TFLite-msg_chn_wacv20-depth-completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model

Ibai Gorordo 2 Oct 04, 2021
A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

Kordel K. France 2 Nov 14, 2022
Continuous Diffusion Graph Neural Network

We present Graph Neural Diffusion (GRAND) that approaches deep learning on graphs as a continuous diffusion process and treats Graph Neural Networks (GNNs) as discretisations of an underlying PDE.

Twitter Research 227 Jan 05, 2023
Cascaded Pyramid Network (CPN) based on Keras (Tensorflow backend)

ML2 Takehome Project Reimplementing the paper: Cascaded Pyramid Network for Multi-Person Pose Estimation Dataset The model uses the COCO dataset which

Vo Van Tu 1 Nov 22, 2021
SSD: Single Shot MultiBox Detector pytorch implementation focusing on simplicity

SSD: Single Shot MultiBox Detector Introduction Here is my pytorch implementation of 2 models: SSD-Resnet50 and SSDLite-MobilenetV2.

Viet Nguyen 149 Jan 07, 2023