Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

Overview

FFD Source Code

Provided is code that demonstrates the training and evaluation of the work presented in the paper: "On the Detection of Digital Face Manipulation" published in CVPR 2020.

The proposed network framework with attention mechanism

Project Webpage

See the MSU CVLab website for project details and access to the DFFD dataset.

http://cvlab.cse.msu.edu/project-ffd.html

Notes

This code is provided as example code, and may not reflect a specific combination of hyper-parameters presented in the paper.

Description of contents

  • xception.py: Defines the Xception network with the attention mechanism
  • train*.py: Train the model on the train data
  • test*.py: Evaluate the model on the test data

Acknowledgements

If you use or refer to this source code, please cite the following paper:

@inproceedings{cvpr2020-dang,
  title={On the Detection of Digital Face Manipulation},
  author={Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, Anil Jain},
  booktitle={In Proceeding of IEEE Computer Vision and Pattern Recognition (CVPR 2020)},
  address={Seattle, WA},
  year={2020}
}
Comments
  • Is it possible to release the script for generating edited images by FaceApp?

    Is it possible to release the script for generating edited images by FaceApp?

    Hi, Thanks for releasing the code and dataset! Part of your dataset is generated by FaceApp (using automated scripts running on android devices). I am wondering if you could also release this android script? I also plan to generate some edited images using FaceApp, and an automated script will be quite helpful!! Thanks!

    opened by zjxgithub 2
  • Question about mask images in dataset

    Question about mask images in dataset

    Thank you for releasing the code and the DFFD dataset!

    I noticed that in the "faceapp" part of the dataset, there is a ground-truth manipulation masks image for each fake image. How are these mask images generated?

    The paper mentioned that the ground-truth manipulation mask were calculated by source images and fake images, but I still did not understand how.

    Thank you for answering my question. :)

    opened by piddnad 2
  • Serveral question about dataset

    Serveral question about dataset

    Thanks for releasing the code and the dataset. I have some questions for the dataset,

    • In align_faces/align_faces.m inside scripts.zip, there is a file called box.txt. But I can't find it anywhere. It seems crucial to align and crop the images.

    image

    • All of the images in dataset are in the resolution of 299x299. I wonder how did you process the images in CelebA. I remember the aligned and cropped image in CelebA are in the resolution of 128x128.
    opened by wheatdog 2
  • attention map and gt mask matching

    attention map and gt mask matching

    Hi, thanks for your work. I have a small question. The attention map size is 19x19, but the gt mask (diff image) is 299x299. Are they matched by downsampling gt mask?

    opened by neverUseThisName 1
  • Are label information leaked in testing process?

    Are label information leaked in testing process?

    Thanks for uploading your code and dataset. After a short view I'm considering your predicting process is like: generating masks with scripts on test data, using test data and their masks to feed into trained model to predict. But I was confused that in your test.py file, you get dataset like this:

    def get_dataset():
      return Dataset('test', BATCH_SIZE, CONFIG['img_size'], CONFIG['map_size'], CONFIG['norms'], SEED)
    

    then you differ masks of real and fake photos by using their labels in dataset.py:

      def __getitem__(self, index):
        im_name = self.images[index]
        img = self.load_image(im_name)
        if self.label_name == 'Real':
          msk = torch.zeros(1,19,19)
        else:
          msk = self.load_mask(im_name.replace('Fake/', 'Mask/'))
        return {'img': img, 'msk': msk, 'lab': self.label, 'im_name': im_name}
    

    Is it fair to distinguish masks by label_name in the testing process? I also wonder how to create Mask/ folder when you predict fake images that donot have corresponding real images?

    If i misunderstand anything please correct me, thanks a lot!

    opened by insomnia1996 0
  • May I know where I can find the imagenet pretrained model?

    May I know where I can find the imagenet pretrained model?

    Hi,

    For using pretrained model: xception-b5690688.pth, may I know where I can find the model specified here: https://github.com/JStehouwer/FFD_CVPR2020/blob/master/xception.py#L243

    Thanks.

    opened by ilovecv 2
  • Error in get_batch in train.py

    Error in get_batch in train.py

    Greetings,

    Many thanks to your wok. I am very interested in your work and I want to try out your model. When I ran the train*.py, I encounter the following issue , here are part of the error messages.

    batch = [next(_.generator, None) for _ in self.datasets]
    

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 91, in self = reduction.pickle.load(from_parent)batch = [next(_.generator, None) for _ in self.datasets]

    File "D:\Fake Detector\attention_map_to_detect_manipulation\FFD_CVPR2020\dataset.py", line 73, in get_batch EOFError: Ran out of input

    and reduction.dump(process_obj, to_child) File "C:\Users\xxx\anaconda3\envs\d2l\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle 'generator' object

    What I did is just make directory data/train/Real(Fake) and place my images dataset into the corresponding folder and then ran the train.py. However, it seems it can't work. May I ask whether I missed anything. I am running the program in windows system and I don't know that will affect as well.

    opened by bitrookie 1
  • Use pretrained model to classify own data?

    Use pretrained model to classify own data?

    Hi @JStehouwer - thank you so much for the awesome code (v2.1)!

    I am trying to use your pretrained model on my own images in order to try out the classifier.

    Are you able to confirm:

    • Filename and format of pretrained model
    • Whether anything else is needed to perform the above classification

    Thanks again

    opened by jtlz2 4
  • dataset questions

    dataset questions

    1、 Whether the published dataset ( FFHQ、FaceAPP、StarGAN、PGGAN、StyleGAN ) has been randomly selected ? And How to generate starGAN mask, how to determine the specific CelebA picture used ? 2、 I have downloaded the FF++、CelebA and DeepFaceLab dataset, how to randomly select the training set, test set and verification set ? And how to set the random seed ? 3、 Which data sets need align processing, and how, please specify ?

    Thank you for your work, it is very good, I will follow your work, but now the problem of dataset makes my work difficult, I hope to get your help.

    opened by miaoct 2
Releases(v2.1)
[BMVC'21] Official PyTorch Implementation of Grounded Situation Recognition with Transformers

Grounded Situation Recognition with Transformers Paper | Model Checkpoint This is the official PyTorch implementation of Grounded Situation Recognitio

Junhyeong Cho 18 Jul 19, 2022
RobustART: Benchmarking Robustness on Architecture Design and Training Techniques

The first comprehensive Robustness investigation benchmark on large-scale dataset ImageNet regarding ARchitecture design and Training techniques towards diverse noises.

132 Dec 23, 2022
A-ESRGAN aims to provide better super-resolution images by using multi-scale attention U-net discriminators.

A-ESRGAN: Training Real-World Blind Super-Resolution with Attention-based U-net Discriminators The authors are hidden for the purpose of double blind

77 Dec 16, 2022
Our VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks.

VMAgent is a platform for exploiting Reinforcement Learning (RL) on Virtual Machine (VM) scheduling tasks. VMAgent is constructed based on one month r

56 Dec 12, 2022
Corgis are the cutest creatures; have 30K of them!

corgi-net This is a dataset of corgi images scraped from the corgi subreddit. After filtering using an ImageNet classifier, the training set consists

Alex Nichol 6 Dec 24, 2022
Pytorch implementation of One-Shot Affordance Detection

One-shot Affordance Detection PyTorch implementation of our one-shot affordance detection models. This repository contains PyTorch evaluation code, tr

46 Dec 12, 2022
New AidForBlind - Various Libraries used like OpenCV and other mentioned in Requirements.txt

AidForBlind Recommended PyCharm IDE Various Libraries used like OpenCV and other

Aalhad Chandewar 1 Jan 13, 2022
Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals.

Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals This repo contains the Pytorch implementation of our paper: Unsupervised Seman

Wouter Van Gansbeke 335 Dec 28, 2022
[ICCV2021] 3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds

3DVG-Transformer This repository is for the ICCV 2021 paper "3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds" Our method "3DV

22 Dec 11, 2022
The ICS Chat System project for NYU Shanghai Fall 2021

ICS_Chat_System [Catenger] This is the ICS Chat System project for NYU Shanghai Fall 2021 Creators: Shavarsh Melikyan, Skyler Chen and Arghya Sarkar,

1 Dec 20, 2021
The code for our paper Semi-Supervised Learning with Multi-Head Co-Training

Semi-Supervised Learning with Multi-Head Co-Training (PyTorch) Abstract Co-training, extended from self-training, is one of the frameworks for semi-su

cmc 6 Dec 04, 2022
Rank 1st in the public leaderboard of ScanRefer (2021-03-18)

InstanceRefer InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referring

63 Dec 07, 2022
Official implementation of the PICASO: Permutation-Invariant Cascaded Attentional Set Operator

PICASO Official PyTorch implemetation for the paper PICASO:Permutation-Invariant Cascaded Attentive Set Operator. Requirements Python 3 torch = 1.0 n

Samira Zare 0 Dec 23, 2021
Source for the paper "Universal Activation Function for machine learning"

Universal Activation Function Tensorflow and Pytorch source code for the paper Yuen, Brosnan, Minh Tu Hoang, Xiaodai Dong, and Tao Lu. "Universal acti

4 Dec 03, 2022
The Multi-Mission Maximum Likelihood framework (3ML)

PyPi Conda The Multi-Mission Maximum Likelihood framework (3ML) A framework for multi-wavelength/multi-messenger analysis for astronomy/astrophysics.

The Multi-Mission Maximum Likelihood (3ML) 62 Dec 30, 2022
This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis Install the package in the requirements.txt, the

108 Dec 23, 2022
The project page of paper: Architecture disentanglement for deep neural networks [ICCV 2021, oral]

This is the project page for the paper: Architecture Disentanglement for Deep Neural Networks, Jie Hu, Liujuan Cao, Tong Tong, Ye Qixiang, ShengChuan

Jie Hu 15 Aug 30, 2022
Flexible Networks for Learning Physical Dynamics of Deformable Objects (2021)

Flexible Networks for Learning Physical Dynamics of Deformable Objects (2021) By Jinhyung Park, Dohae Lee, In-Kwon Lee from Yonsei University (Seoul,

Jinhyung Park 0 Jan 09, 2022
An unofficial personal implementation of UM-Adapt, specifically to tackle joint estimation of panoptic segmentation and depth prediction for autonomous driving datasets.

Semisupervised Multitask Learning This repository is an unofficial and slightly modified implementation of UM-Adapt[1] using PyTorch. This code primar

Abhinav Atrishi 11 Nov 25, 2022
Weakly Supervised Learning of Rigid 3D Scene Flow

Weakly Supervised Learning of Rigid 3D Scene Flow This repository provides code and data to train and evaluate a weakly supervised method for rigid 3D

Zan Gojcic 124 Dec 27, 2022