Fake videos detection by tracing the source using video hashing retrieval.

Last update: Dec 22, 2022

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

Introduction
Train or Test
Tracing
- Trace Samples
Localization
- Localization Samples

✨ Introduction

VTL

Video Tracing and Tampering Localization (VTL). A novel framework to detect fake video (clipping, cropping, blur, etc.) by tracing the source video of fake video. 1) Training hash centers as HCs. 2) Finding index of source video from HCs. 3) Masking the different between fake video and source video as a result of comparison (auxiliary information).

Trace Samples and Acc of HashBits

Although the source videos are very similar, we can accurately find the source videos of the fake video clips.

DFTL Dataset Samples

Same person with different scenes. You can download full 16 minutes videos of source video and fake video by follows link.

Source Videos

Fake Videos

Different fake videos from same source.

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

The first gif of boat is source video, and remaining five videos generated by different inpainting methods.

🔬 Train or Test

Datasets Download

BaiduNetdisk code：VTLs

actors: Source videos and fake videos of full 16 minutes. You can use these videos to make richer datasets.
DFTL: Dataset of DFTL, the DFTL build from actors.
DAVIS2016-TL: Extension of DAVIS2016

Extract to the same directory as the code (vtl). Example:

├─other files
├─project
│  ├─vrf: dataset of DFTL
│  ├─inpainting: dataset of DAVIS2016-TL
│  └─vtl: our code
│      ├─CSQ: Central Similarity Quantization for Efficient Image and Video Retrieval
│      ├─dmac: Compared method of Localization
│      └─codes

Train

Pretrained models and hash centers

pip install -r requirements.txt

Model	DFTL	DAVIS2016-TL
ViTHash	64-1024bits	64-1024bits
Generator	link	link

Parameters

local_rank: gpu id
path: dataset path
type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting

Train ViTHash

python train_h.py --local_rank=0 --path=../vrf --type=0 --bits=128

Train Generator

python train_g.py --local_rank=0 --path=../vrf --type=0

Test

Test IOU

The test script will test Generator of VTL and DMAC together on DFTL and DAVIS2016-TL. You can modify it for yourself.

python test_iou.py

Test ViTHash

type: choice dataloader
- 0: DFTL dataloader, dir name is vrf
- 1: DAVIS2016-TL dataloader, dir name is inpainting
path: dataset path
hashbits: 128 256 512 or 1024, will load different pre-trained model and hash JSON file.

python test.py 1 ../inpainting 512

Test CSQ

cd ./CSQ
run test script

python hash_test_vrf.py --dataset=Inpainting --pretrained_3d=./Inpainting_64bits.pth

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Deep Unsupervised Image Hashing by Maximizing Bit Entropy This is the PyTorch implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hash

62 Dec 30, 2022

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

71 Dec 22, 2022

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021) Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMill

23 Dec 9, 2022

Enhancing Knowledge Tracing via Adversarial Training

Enhancing Knowledge Tracing via Adversarial Training This repository contains source code for the paper "Enhancing Knowledge Tracing via Adversarial T

14 Oct 24, 2022

Implementation of light baking system for ray tracing based on Activision's UberBake

Vulkan Light Bakary MSU Graphics Group Student's Diploma Project Treefonov Andrey [GitHub] [LinkedIn] Project Goal The goal of the project is to imple

7 Dec 27, 2022

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

TensorGeodesic Ray tracing of a Schwarzschild black hole written entirely in TensorFlow. Dependencies: Python 3 TensorFlow 2.x numpy matplotlib About

5 Jan 15, 2022

Space-event-trace - Tracing service for spaceteam events

space-event-trace Tracing service for TU Wien Spaceteam events. This service is

2 Jan 4, 2022

Activity image-based video retrieval

Cross-modal-retrieval Our approach is focus on Activity Image-to-Video Retrieval (AIVR) task. The compared methods are state-of-the-art single modalit

75 Oct 21, 2021

A Joint Video and Image Encoder for End-to-End Retrieval

Frozen️ in Time ❄️ ️️️️ ⏳ A Joint Video and Image Encoder for End-to-End Retrieval project page | arXiv | webvid-data Repository containing the code,

225 Dec 25, 2022

Comments

the pre-trained model

I downloaded the provided pre-trained model and the test acc in DFTL and DAVIS2016-TL reached 100%, is this correct? Why is it so much higher than in the article?

opened by ymhzyj 1

Fake videos detection by tracing the source using video hashing retrieval.

Related tags

Overview

Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos 🎉️

📜 Directory

✨ Introduction

VTL

Trace Samples and Acc of HashBits

DFTL Dataset Samples

Source Videos

Fake Videos

Source Video

Fake Videos of Different Face Swap Methods

DAVIS2016-TL Dataset Samples

🔬 Train or Test

Datasets Download

Train

Test

🚀️ Tracing

Trace Samples

👀️ Localization

Localization Samples

DAVIS2016-TL

DFTL

You might also like...

Implementation of accepted AAAI 2021 paper: Deep Unsupervised Image Hashing by Maximizing Bit Entropy

Official implementation of NeurIPS 2021 paper "One Loss for All: Deep Hashing with a Single Cosine Similarity based Learning Objective"

Tracing Versus Freehand for Evaluating Computer-Generated Drawings (SIGGRAPH 2021)

Enhancing Knowledge Tracing via Adversarial Training

Implementation of light baking system for ray tracing based on Activision's UberBake

Ray tracing of a Schwarzschild black hole written entirely in TensorFlow.

Space-event-trace - Tracing service for spaceteam events

Activity image-based video retrieval

A Joint Video and Image Encoder for End-to-End Retrieval

Comments

the pre-trained model

Releases(v1.0)

Owner

A sketch extractor for anime/illustration.

Beyond imagenet attack (accepted by ICLR 2022) towards crafting adversarial examples for black-box domains.

The source code of the paper "SHGNN: Structure-Aware Heterogeneous Graph Neural Network"

Official repository for HOTR: End-to-End Human-Object Interaction Detection with Transformers (CVPR'21, Oral Presentation)

Implementation of fast algorithms for Maximum Spanning Tree (MST) parsing that includes fast ArcMax+Reweighting+Tarjan algorithm for single-root dependency parsing.

Inflated i3d network with inception backbone, weights transfered from tensorflow

BuildingNet: Learning to Label 3D Buildings

Minimal But Practical Image Classifier Pipline Using Pytorch, Finetune on ResNet18, Got 99% Accuracy on Own Small Datasets.

Deploy a ML inference service on a budget in less than 10 lines of code.

PyTorch implementation of Barlow Twins.

The project was to detect traffic signs, based on the Megengine framework.

PyTorch implementation of saliency map-aided GAN for Auto-demosaic+denosing

git《Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser》(2021) GitHub: [fig5]

Pytorch GUI(demo) for iVOS(interactive VOS) and GIS (Guided iVOS)

ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure

iBOT: Image BERT Pre-Training with Online Tokenizer

Code associated with the paper "Deep Optics for Single-shot High-dynamic-range Imaging"

Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

Fast SHAP value computation for interpreting tree-based models

The final project of "Applying AI to EHR Data" of "AI for Healthcare" nanodegree - Udacity.