Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Overview

DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification (2021) by Hai Phan and Anh Nguyen.

If you use this software, please consider citing:

@article{hai2021deepface,
  title={DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification},
  author={Hai Phan, Anh Nguyen},
  journal={arXiv preprint arXiv:2112.04016},
  year={2021}
}

1. Requirements

Python >= 3.5
Pytorch > 1.0
Opencv >= 3.4.4
pip install tqmd

2. Download datasets and pretrained models

  1. Download LFW, out-of-distribution (OOD) LFW test sets, and pretrained models: Google Drive

  2. Create the following folders:

mkdir data
mkdir pretrained
  1. Extract LFW datasets (e.g. lfw_crop_96x112.tar.gz) to data/
  2. Copy models (e.g. resnet18_110.pth) to pretrained/

3. How to run

3.1 Run examples

  • Run testing LFW images

    • -mask, -sunglass, -crop: flags for using corresponding OOD query images (i.e., faces with masks or sunglasses or randomly-cropped images).
    bash run_test.sh
    
  • Run demo: The demo gives results of top-5 images of stage 1 and stage 2 (including flow visualization of EMD).

    • -mask: image retrieval using a masked-face query image given a gallery of normal LFW images.
    • -sunglass and -crop: similar to the setup of -mask.
    • The results will be saved in the results/demo directory.
    bash run_demo.sh
    
  • Run retrieval using the full LFW gallery

    • Set the argument args.data_folder to data in .sh files.

3.2 Reproduce results

  • Make sure lfw-align-128 and lfw-align-128-crop70 dataset in data/ directory (e.g. data/lfw-align-128-crop70), ArcFace [2] model resnet18_110.pth in pretrained/ directory (e.g. pretrained/resnet18_110.pth). Run the following commands to reproduce the Table 1 results in our paper.

    • Arguments:

      • Methods can be apc, uniform, or sc
      • -l: 4 or 8 for 4x4 and 8x8 respectively.
      • -a: alpha parameter mentioned in the paper.
    • Normal LFW with 1680 classes:

    python test_face.py -method apc -fm arcface -d lfw_1680 -a -1 -data_folder data -l 4
    
    • LFW-crop:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -crop 
    
    • Note: The full LFW dataset have 5,749 people for a total of 13,233 images; however, only 1,680 people have two or more images (See LFW for details). However, in our normal LFW dataset, the identical images will not be considered in face identification. So, the difference between lfw and lfw_1680 is that the lfw setup uses the full LFW (including people with a single image) but the lfw_1680 uses only 1,680 people who have two or more images.
  • For other OOD datasets, run the following command:

    • LFW-mask:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -mask 
    
    • LFW-sunglass:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -sunglass 
    

3.3 Run visualization with two images

python visualize_faces.py -method [methods] -fm [face models] -model_path [model dir] -in1 [1st image] -in2 [2nd image] -weight [1/0: showing weight heatmaps] 

The results are in results/flow and results/heatmap (if -weight flag is on).

3.4 Use your own images

  1. Facial alignment. See align_face.py for details.
pip install scikit-image
pip install face-alignment
  • For making face alignment with size of 160x160 for Arcface (128x128) and FaceNet (160x160), the reference points are as follow (see function alignment in align_face.py).
ref_pts = [ [61.4356, 54.6963],[118.5318, 54.6963], [93.5252, 90.7366],[68.5493, 122.3655],[110.7299, 122.3641]]
crop_size = (160, 160)
  1. Create a folder including all persons (folders: name of person) and put it to '/data'
  2. Create a txt file with format: [image_path],[label] of that folder (See lfw file for details)
  3. Modify face loader: Add your txt file in function: get_face_dataloader.

4. License

MIT

5. References

  1. W. Zhao, Y. Rao, Z. Wang, J. Lu, Zhou. Towards interpretable deep metric learning with structural matching, ICCV 2021 DIML
  2. J. Deng, J. Guo, X. Niannan, and StefanosZafeiriou. Arcface: Additive angular margin loss for deepface recognition, CVPR 2019 Arcface Pytorch
  3. H. Wang, Y. Wang, Z. Zhou, X. Ji, DihongGong, J. Zhou, Z. Li, W. Liu. Cosface: Large margin cosine loss for deep face recognition, CVPR 2018 CosFace Pytorch
  4. F. Schroff, D. Kalenichenko, J. Philbin. Facenet: A unified embedding for face recognition and clustering. CVPR 2015 FaceNet Pytorch
  5. L. Weiyang, W. Yandong, Y. Zhiding, L. Ming, R. Bhiksha, S. Le. SphereFace: Deep Hypersphere Embedding for Face Recognition, CVPR 2017 sphereface, sphereface pytorch
  6. Chi Zhang, Yujun Cai, Guosheng Lin, Chunhua Shen. Deepemd: Differentiable earth mover’s distance for few-shotlearning, CVPR 2020 paper
Owner
Anh M. Nguyen
Learning in the deep...
Anh M. Nguyen
JAXDL: JAX (Flax) Deep Learning Library

JAXDL: JAX (Flax) Deep Learning Library Simple and clean JAX/Flax deep learning algorithm implementations: Soft-Actor-Critic (arXiv:1812.05905) Transf

Patrick Hart 4 Nov 27, 2022
Improving XGBoost survival analysis with embeddings and debiased estimators

xgbse: XGBoost Survival Embeddings "There are two cultures in the use of statistical modeling to reach conclusions from data

Loft 242 Dec 30, 2022
SegNet including indices pooling for Semantic Segmentation with tensorflow and keras

SegNet SegNet is a model of semantic segmentation based on Fully Comvolutional Network. This repository contains the implementation of learning and te

Yuta Kamikawa 172 Dec 23, 2022
基于Paddlepaddle复现yolov5,支持PaddleDetection接口

PaddleDetection yolov5 https://github.com/Sharpiless/PaddleDetection-Yolov5 简介 PaddleDetection飞桨目标检测开发套件,旨在帮助开发者更快更好地完成检测模型的组建、训练、优化及部署等全开发流程。 PaddleD

36 Jan 07, 2023
Kernel Point Convolutions

Created by Hugues THOMAS Introduction Update 27/04/2020: New PyTorch implementation available. With SemanticKitti, and Windows supported. This reposit

Hugues THOMAS 584 Jan 07, 2023
This is the implementation of the paper "Self-supervised Outdoor Scene Relighting"

Self-supervised Outdoor Scene Relighting This is the implementation of the paper "Self-supervised Outdoor Scene Relighting". The model is implemented

Ye Yu 24 Dec 17, 2022
Unimodal Face Classification with Multimodal Training

Unimodal Face Classification with Multimodal Training This is a PyTorch implementation of the following paper: Unimodal Face Classification with Multi

Wenbin Teng 3 Jul 06, 2022
Management Dashboard for Torchserve

Torchserve Dashboard Torchserve Dashboard using Streamlit Related blog post Usage Additional Requirement: torchserve (recommended:v0.5.2) Simply run:

Ceyda Cinarel 103 Dec 10, 2022
Yolo Traffic Light Detection With Python

Yolo-Traffic-Light-Detection This project is based on detecting the Traffic light. Pretained data is used. This application entertained both real time

Ananta Raj Pant 2 Aug 08, 2022
Motion Reconstruction Code and Data for Skills from Videos (SFV)

Motion Reconstruction Code and Data for Skills from Videos (SFV) This repo contains the data and the code for motion reconstruction component of the S

268 Dec 01, 2022
pytorch implementation of Attention is all you need

A Pytorch Implementation of the Transformer: Attention Is All You Need Our implementation is largely based on Tensorflow implementation Requirements N

230 Dec 07, 2022
Tensorflow python implementation of "Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos"

Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos This repository is the official tensorflow python implementation

Yasamin Jafarian 287 Jan 06, 2023
An open framework for Federated Learning.

Welcome to Intel® Open Federated Learning Federated learning is a distributed machine learning approach that enables organizations to collaborate on m

Intel Corporation 397 Dec 27, 2022
Pre-trained models for a Cascaded-FCN in caffe and tensorflow that segments

Cascaded-FCN This repository contains the pre-trained models for a Cascaded-FCN in caffe and tensorflow that segments the liver and its lesions out of

300 Nov 22, 2022
This repository contains PyTorch code for Robust Vision Transformers.

This repository contains PyTorch code for Robust Vision Transformers.

117 Dec 07, 2022
This repository includes code of my study about Asynchronous in Frequency domain of GAN images.

Exploring the Asynchronous of the Frequency Spectra of GAN-generated Facial Images Binh M. Le & Simon S. Woo, "Exploring the Asynchronous of the Frequ

4 Aug 06, 2022
Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

RTM3D-PyTorch The PyTorch Implementation of the paper: RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (ECCV 2020

Nguyen Mau Dzung 271 Nov 29, 2022
Pytorch re-implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition (CVPR 2022)

SwinTextSpotter This is the pytorch implementation of Paper: SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text R

mxin262 183 Jan 03, 2023
Fuwa-http - The http client implementation for the fuwa eco-system

Fuwa HTTP The HTTP client implementation for the fuwa eco-system Example import

Fuwa 2 Feb 16, 2022
PyTorch implementation of Deformable Convolution

PyTorch implementation of Deformable Convolution !!!Warning: There is some issues in this implementation and this repo is not maintained any more, ple

Wei Ouyang 893 Dec 18, 2022