Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Overview

DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification (2021) by Hai Phan and Anh Nguyen.

If you use this software, please consider citing:

@article{hai2021deepface,
  title={DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification},
  author={Hai Phan, Anh Nguyen},
  journal={arXiv preprint arXiv:2112.04016},
  year={2021}
}

1. Requirements

Python >= 3.5
Pytorch > 1.0
Opencv >= 3.4.4
pip install tqmd

2. Download datasets and pretrained models

  1. Download LFW, out-of-distribution (OOD) LFW test sets, and pretrained models: Google Drive

  2. Create the following folders:

mkdir data
mkdir pretrained
  1. Extract LFW datasets (e.g. lfw_crop_96x112.tar.gz) to data/
  2. Copy models (e.g. resnet18_110.pth) to pretrained/

3. How to run

3.1 Run examples

  • Run testing LFW images

    • -mask, -sunglass, -crop: flags for using corresponding OOD query images (i.e., faces with masks or sunglasses or randomly-cropped images).
    bash run_test.sh
    
  • Run demo: The demo gives results of top-5 images of stage 1 and stage 2 (including flow visualization of EMD).

    • -mask: image retrieval using a masked-face query image given a gallery of normal LFW images.
    • -sunglass and -crop: similar to the setup of -mask.
    • The results will be saved in the results/demo directory.
    bash run_demo.sh
    
  • Run retrieval using the full LFW gallery

    • Set the argument args.data_folder to data in .sh files.

3.2 Reproduce results

  • Make sure lfw-align-128 and lfw-align-128-crop70 dataset in data/ directory (e.g. data/lfw-align-128-crop70), ArcFace [2] model resnet18_110.pth in pretrained/ directory (e.g. pretrained/resnet18_110.pth). Run the following commands to reproduce the Table 1 results in our paper.

    • Arguments:

      • Methods can be apc, uniform, or sc
      • -l: 4 or 8 for 4x4 and 8x8 respectively.
      • -a: alpha parameter mentioned in the paper.
    • Normal LFW with 1680 classes:

    python test_face.py -method apc -fm arcface -d lfw_1680 -a -1 -data_folder data -l 4
    
    • LFW-crop:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -crop 
    
    • Note: The full LFW dataset have 5,749 people for a total of 13,233 images; however, only 1,680 people have two or more images (See LFW for details). However, in our normal LFW dataset, the identical images will not be considered in face identification. So, the difference between lfw and lfw_1680 is that the lfw setup uses the full LFW (including people with a single image) but the lfw_1680 uses only 1,680 people who have two or more images.
  • For other OOD datasets, run the following command:

    • LFW-mask:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -mask 
    
    • LFW-sunglass:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -sunglass 
    

3.3 Run visualization with two images

python visualize_faces.py -method [methods] -fm [face models] -model_path [model dir] -in1 [1st image] -in2 [2nd image] -weight [1/0: showing weight heatmaps] 

The results are in results/flow and results/heatmap (if -weight flag is on).

3.4 Use your own images

  1. Facial alignment. See align_face.py for details.
pip install scikit-image
pip install face-alignment
  • For making face alignment with size of 160x160 for Arcface (128x128) and FaceNet (160x160), the reference points are as follow (see function alignment in align_face.py).
ref_pts = [ [61.4356, 54.6963],[118.5318, 54.6963], [93.5252, 90.7366],[68.5493, 122.3655],[110.7299, 122.3641]]
crop_size = (160, 160)
  1. Create a folder including all persons (folders: name of person) and put it to '/data'
  2. Create a txt file with format: [image_path],[label] of that folder (See lfw file for details)
  3. Modify face loader: Add your txt file in function: get_face_dataloader.

4. License

MIT

5. References

  1. W. Zhao, Y. Rao, Z. Wang, J. Lu, Zhou. Towards interpretable deep metric learning with structural matching, ICCV 2021 DIML
  2. J. Deng, J. Guo, X. Niannan, and StefanosZafeiriou. Arcface: Additive angular margin loss for deepface recognition, CVPR 2019 Arcface Pytorch
  3. H. Wang, Y. Wang, Z. Zhou, X. Ji, DihongGong, J. Zhou, Z. Li, W. Liu. Cosface: Large margin cosine loss for deep face recognition, CVPR 2018 CosFace Pytorch
  4. F. Schroff, D. Kalenichenko, J. Philbin. Facenet: A unified embedding for face recognition and clustering. CVPR 2015 FaceNet Pytorch
  5. L. Weiyang, W. Yandong, Y. Zhiding, L. Ming, R. Bhiksha, S. Le. SphereFace: Deep Hypersphere Embedding for Face Recognition, CVPR 2017 sphereface, sphereface pytorch
  6. Chi Zhang, Yujun Cai, Guosheng Lin, Chunhua Shen. Deepemd: Differentiable earth mover’s distance for few-shotlearning, CVPR 2020 paper
Owner
Anh M. Nguyen
Learning in the deep...
Anh M. Nguyen
A Pytorch loader for MVTecAD dataset.

MVTecAD A Pytorch loader for MVTecAD dataset. It strictly follows the code style of common Pytorch datasets, such as torchvision.datasets.CIFAR10. The

Jiyuan 1 Dec 27, 2021
Unofficial implementation of the Involution operation from CVPR 2021

involution_pytorch Unofficial PyTorch implementation of "Involution: Inverting the Inherence of Convolution for Visual Recognition" by Li et al. prese

Rishabh Anand 46 Dec 07, 2022
LiDAR R-CNN: An Efficient and Universal 3D Object Detector

LiDAR R-CNN: An Efficient and Universal 3D Object Detector Introduction This is the official code of LiDAR R-CNN: An Efficient and Universal 3D Object

TuSimple 295 Jan 05, 2023
This repository contains a Ruby API for utilizing TensorFlow.

tensorflow.rb Description This repository contains a Ruby API for utilizing TensorFlow. Linux CPU Linux GPU PIP Mac OS CPU Not Configured Not Configur

somatic labs 825 Dec 26, 2022
Code for the paper "Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness"

DU-VAE This is the pytorch implementation of the paper "Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness" Acknowledgement

Dazhong Shen 4 Oct 19, 2022
GndNet: Fast ground plane estimation and point cloud segmentation for autonomous vehicles using deep neural networks.

GndNet: Fast Ground plane Estimation and Point Cloud Segmentation for Autonomous Vehicles. Authors: Anshul Paigwar, Ozgur Erkent, David Sierra Gonzale

Anshul Paigwar 114 Dec 29, 2022
TensorFlow implementation of ENet

TensorFlow-ENet TensorFlow implementation of ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. This model was tested on th

Kwotsin 255 Oct 17, 2022
This repository allows the user to automatically scale a 3D model/mesh/point cloud on Agisoft Metashape

Metashape-Utils This repository allows the user to automatically scale a 3D model/mesh/point cloud on Agisoft Metashape, given a set of 2D coordinates

INSCRIBE 4 Nov 07, 2022
An implementation for the ICCV 2021 paper Deep Permutation Equivariant Structure from Motion.

Deep Permutation Equivariant Structure from Motion Paper | Poster This repository contains an implementation for the ICCV 2021 paper Deep Permutation

72 Dec 27, 2022
This repository contains python code necessary to replicated the experiments performed in our paper "Invariant Ancestry Search"

InvariantAncestrySearch This repository contains python code necessary to replicated the experiments performed in our paper "Invariant Ancestry Search

Phillip Bredahl Mogensen 0 Feb 02, 2022
Vector Quantized Diffusion Model for Text-to-Image Synthesis

Vector Quantized Diffusion Model for Text-to-Image Synthesis Due to company policy, I have to set microsoft/VQ-Diffusion to private for now, so I prov

Shuyang Gu 294 Jan 05, 2023
Public repository containing materials used for Feed Forward (FF) Neural Networks article.

Art041_NN_Feed_Forward Public repository containing materials used for Feed Forward (FF) Neural Networks article. -- Illustration of a very simple Fee

SolClover 2 Dec 29, 2021
g9.py - Torch interactive graphics

g9.py - Torch interactive graphics A Torch toy in the browser. Demo at https://srush.github.io/g9py/ This is a shameless copy of g9.js, written in Pyt

Sasha Rush 13 Nov 16, 2022
A MNIST-like fashion product database. Benchmark

Fashion-MNIST Table of Contents Why we made Fashion-MNIST Get the Data Usage Benchmark Visualization Contributing Contact Citing Fashion-MNIST License

Zalando Research 10.5k Jan 08, 2023
"Segmenter: Transformer for Semantic Segmentation" reproduced via mmsegmentation

Segmenter-based-on-OpenMMLab "Segmenter: Transformer for Semantic Segmentation, arxiv 2105.05633." reproduced via mmsegmentation. We reproduce Segment

EricKani 22 Feb 24, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

49 Nov 23, 2022
Unofficial PyTorch Implementation of "Augmenting Convolutional networks with attention-based aggregation"

Pytorch Implementation of Augmenting Convolutional networks with attention-based aggregation This is the unofficial PyTorch Implementation of "Augment

DK 20 Sep 09, 2022
Code for the Higgs Boson Machine Learning Challenge organised by CERN & EPFL

A method to solve the Higgs boson challenge using Least Squares - Novae This project is the Project 1 of EPFL CS-433 Machine Learning. The project is

Giacomo Orsi 1 Nov 09, 2021
Reinforcement Learning for Portfolio Management

qtrader Reinforcement Learning for Portfolio Management Why Reinforcement Learning? Learns the optimal action, rather than models the market. Adaptive

Angelos Filos 406 Jan 01, 2023
Segmentation Training Pipeline

Segmentation Training Pipeline This package is a part of Musket ML framework. Reasons to use Segmentation Pipeline Segmentation Pipeline was developed

Musket ML 52 Dec 12, 2022