Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Overview

DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification

Official Implementation for the paper DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification (2021) by Hai Phan and Anh Nguyen.

If you use this software, please consider citing:

@article{hai2021deepface,
  title={DeepFace-EMD: Re-ranking Using Patch-wise Earth Mover’s Distance Improves Out-Of-Distribution Face Identification},
  author={Hai Phan, Anh Nguyen},
  journal={arXiv preprint arXiv:2112.04016},
  year={2021}
}

1. Requirements

Python >= 3.5
Pytorch > 1.0
Opencv >= 3.4.4
pip install tqmd

2. Download datasets and pretrained models

  1. Download LFW, out-of-distribution (OOD) LFW test sets, and pretrained models: Google Drive

  2. Create the following folders:

mkdir data
mkdir pretrained
  1. Extract LFW datasets (e.g. lfw_crop_96x112.tar.gz) to data/
  2. Copy models (e.g. resnet18_110.pth) to pretrained/

3. How to run

3.1 Run examples

  • Run testing LFW images

    • -mask, -sunglass, -crop: flags for using corresponding OOD query images (i.e., faces with masks or sunglasses or randomly-cropped images).
    bash run_test.sh
    
  • Run demo: The demo gives results of top-5 images of stage 1 and stage 2 (including flow visualization of EMD).

    • -mask: image retrieval using a masked-face query image given a gallery of normal LFW images.
    • -sunglass and -crop: similar to the setup of -mask.
    • The results will be saved in the results/demo directory.
    bash run_demo.sh
    
  • Run retrieval using the full LFW gallery

    • Set the argument args.data_folder to data in .sh files.

3.2 Reproduce results

  • Make sure lfw-align-128 and lfw-align-128-crop70 dataset in data/ directory (e.g. data/lfw-align-128-crop70), ArcFace [2] model resnet18_110.pth in pretrained/ directory (e.g. pretrained/resnet18_110.pth). Run the following commands to reproduce the Table 1 results in our paper.

    • Arguments:

      • Methods can be apc, uniform, or sc
      • -l: 4 or 8 for 4x4 and 8x8 respectively.
      • -a: alpha parameter mentioned in the paper.
    • Normal LFW with 1680 classes:

    python test_face.py -method apc -fm arcface -d lfw_1680 -a -1 -data_folder data -l 4
    
    • LFW-crop:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -crop 
    
    • Note: The full LFW dataset have 5,749 people for a total of 13,233 images; however, only 1,680 people have two or more images (See LFW for details). However, in our normal LFW dataset, the identical images will not be considered in face identification. So, the difference between lfw and lfw_1680 is that the lfw setup uses the full LFW (including people with a single image) but the lfw_1680 uses only 1,680 people who have two or more images.
  • For other OOD datasets, run the following command:

    • LFW-mask:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -mask 
    
    • LFW-sunglass:
    python test_face.py -method apc -fm arcface -d lfw -a 0.7 -data_folder data -l 4 -sunglass 
    

3.3 Run visualization with two images

python visualize_faces.py -method [methods] -fm [face models] -model_path [model dir] -in1 [1st image] -in2 [2nd image] -weight [1/0: showing weight heatmaps] 

The results are in results/flow and results/heatmap (if -weight flag is on).

3.4 Use your own images

  1. Facial alignment. See align_face.py for details.
pip install scikit-image
pip install face-alignment
  • For making face alignment with size of 160x160 for Arcface (128x128) and FaceNet (160x160), the reference points are as follow (see function alignment in align_face.py).
ref_pts = [ [61.4356, 54.6963],[118.5318, 54.6963], [93.5252, 90.7366],[68.5493, 122.3655],[110.7299, 122.3641]]
crop_size = (160, 160)
  1. Create a folder including all persons (folders: name of person) and put it to '/data'
  2. Create a txt file with format: [image_path],[label] of that folder (See lfw file for details)
  3. Modify face loader: Add your txt file in function: get_face_dataloader.

4. License

MIT

5. References

  1. W. Zhao, Y. Rao, Z. Wang, J. Lu, Zhou. Towards interpretable deep metric learning with structural matching, ICCV 2021 DIML
  2. J. Deng, J. Guo, X. Niannan, and StefanosZafeiriou. Arcface: Additive angular margin loss for deepface recognition, CVPR 2019 Arcface Pytorch
  3. H. Wang, Y. Wang, Z. Zhou, X. Ji, DihongGong, J. Zhou, Z. Li, W. Liu. Cosface: Large margin cosine loss for deep face recognition, CVPR 2018 CosFace Pytorch
  4. F. Schroff, D. Kalenichenko, J. Philbin. Facenet: A unified embedding for face recognition and clustering. CVPR 2015 FaceNet Pytorch
  5. L. Weiyang, W. Yandong, Y. Zhiding, L. Ming, R. Bhiksha, S. Le. SphereFace: Deep Hypersphere Embedding for Face Recognition, CVPR 2017 sphereface, sphereface pytorch
  6. Chi Zhang, Yujun Cai, Guosheng Lin, Chunhua Shen. Deepemd: Differentiable earth mover’s distance for few-shotlearning, CVPR 2020 paper
Owner
Anh M. Nguyen
Learning in the deep...
Anh M. Nguyen
Network Compression via Central Filter

Network Compression via Central Filter Environments The code has been tested in the following environments: Python 3.8 PyTorch 1.8.1 cuda 10.2 torchsu

2 May 12, 2022
POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propagation including diffraction

POPPY: Physical Optics Propagation in Python POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propaga

Space Telescope Science Institute 132 Dec 15, 2022
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

chitra What is chitra? chitra (चित्र) is a multi-functional library for full-stack Deep Learning. It simplifies Model Building, API development, and M

Aniket Maurya 210 Dec 21, 2022
Simple image captioning model - CLIP prefix captioning.

Simple image captioning model - CLIP prefix captioning.

688 Jan 04, 2023
Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Claims.

MTM This is the official repository of the paper: Article Reranking by Memory-enhanced Key Sentence Matching for Detecting Previously Fact-checked Cla

ICTMCG 13 Sep 17, 2022
Cleaned test data list of DukeMTMC-reID, ICCV2021

Cleaned DukeMTMC-reID Cleaned data list of DukeMTMC-reID released with our paper accepted by ICCV 2021: Learning Instance-level Spatial-Temporal Patte

14 Feb 19, 2022
A pytorch reprelication of the model-based reinforcement learning algorithm MBPO

Overview This is a re-implementation of the model-based RL algorithm MBPO in pytorch as described in the following paper: When to Trust Your Model: Mo

Xingyu Lin 93 Jan 05, 2023
FairyTailor: Multimodal Generative Framework for Storytelling

FairyTailor: Multimodal Generative Framework for Storytelling

Eden Bens 172 Dec 30, 2022
A2LP for short, ECCV2020 spotlight, Investigating SSL principles for UDA problems

Label-Propagation-with-Augmented-Anchors (A2LP) Official codes of the ECCV2020 spotlight (label propagation with augmented anchors: a simple semi-supe

20 Oct 27, 2022
Playable Video Generation

Playable Video Generation Playable Video Generation Willi Menapace, Stéphane Lathuilière, Sergey Tulyakov, Aliaksandr Siarohin, Elisa Ricci Paper: ArX

Willi Menapace 136 Dec 31, 2022
Pytorch implementation of winner from VQA Chllange Workshop in CVPR'17

2017 VQA Challenge Winner (CVPR'17 Workshop) pytorch implementation of Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challeng

Mark Dong 166 Dec 11, 2022
Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

wechat-wow-analysis Understanding WeChat User Preferences and “Wow” Diffusion. Fanjin Zhang, Jie Tang, Xueyi Liu, Zhenyu Hou, Yuxiao Dong, Jing Zhang,

18 Sep 16, 2022
Contrastive Feature Loss for Image Prediction

Contrastive Feature Loss for Image Prediction We provide a PyTorch implementation of our contrastive feature loss presented in: Contrastive Feature Lo

Alex Andonian 44 Oct 05, 2022
Evaluation toolkit of the informative tracking benchmark comprising 9 scenarios, 180 diverse videos, and new challenges.

Informative-tracking-benchmark Informative tracking benchmark (ITB) higher diversity. It contains 9 representative scenarios and 180 diverse videos. m

Xin Li 15 Nov 26, 2022
Transformer in Vision

Transformer-in-Vision Recent Transformer-based CV and related works. Welcome to comment/contribute! Keep updated. Resource SCENIC: A JAX Library for C

Yong-Lu Li 1.1k Dec 30, 2022
A list of all papers and resoureces on Semantic Segmentation

Semantic-Segmentation A list of all papers and resoureces on Semantic Segmentation. Dataset importance SemanticSegmentation_DL Some implementation of

Alan Tang 1.1k Dec 12, 2022
Making Structure-from-Motion (COLMAP) more robust to symmetries and duplicated structures

SfM disambiguation with COLMAP About Structure-from-Motion generally fails when the scene exhibits symmetries and duplicated structures. In this repos

Computer Vision and Geometry Lab 193 Dec 26, 2022
PyTorch implementation for paper Neural Marching Cubes.

NMC PyTorch implementation for paper Neural Marching Cubes, Zhiqin Chen, Hao Zhang. Paper | Supplementary Material (to be updated) Citation If you fin

Zhiqin Chen 109 Dec 27, 2022
chen2020iros: Learning an Overlap-based Observation Model for 3D LiDAR Localization.

Overlap-based 3D LiDAR Monte Carlo Localization This repo contains the code for our IROS2020 paper: Learning an Overlap-based Observation Model for 3D

Photogrammetry & Robotics Bonn 219 Dec 15, 2022
Auto-updating data to assist in investment to NEPSE

Symbol Ratios Summary Sector LTP Undervalued Bonus % MEGA Strong Commercial Banks 368 5 10 JBBL Strong Development Banks 568 5 10 SIFC Strong Finance

Amit Chaudhary 16 Nov 01, 2022