Corruption Invariant Learning for Re-identification

Overview

Corruption Invariant Learning for Re-identification

The official repository for Benchmarks for Corruption Invariant Person Re-identification (NeurIPS 2021 Track on Datasets and Benchmarks), with exhaustive study on corruption invariant learning in single- and cross-modality ReID datasets, including Market-1501-C, CUHK03-C, MSMT17-C, SYSU-MM01-C, RegDB-C.

PWC PWC PWC PWC PWC

Maintenance Plan

The benchmark will be maintained by the authors. We will get constant lectures about the new proposed ReID models and evaluate them under the CIL benchmark settings in time. Besides, we gladly take feedback to the CIL benchmark and welcome any contributions in terms of the new ReID models and corresponding evaluations. Please feel free to contact us, [email protected] .

TODO:

  • other datasets configurations
  • get started tutorial
  • more detailed statistical evaluations
  • checkpoints of the baseline models
  • cross-modality preson Re-ID dataset, CUHK-PEDES
  • other ReID datasets, like VehicleID, VeRi-776, etc.

(Note: codebase from TransReID)

Quick Start

1. Install dependencies

  • python=3.7.0
  • pytorch=1.6.0
  • torchvision=0.7.0
  • timm=0.4.9
  • albumentations=0.5.2
  • imagecorruptions=1.1.2
  • h5py=2.10.0
  • cython=0.29.24
  • yacs=0.1.6

2. Prepare dataset

Download the datasets, Market-1501, CUHK03, MSMT17. Set the root path of the dataset in congigs/Market/resnet_base.yml, DATASETS: ROOT_DIR: ('root'), or set it in scripts/train_market.sh, DATASETS.ROOT_DIR "('root')".

3. Train

Train a CIL model on Market-1501,

sh ./scripts/train_market.sh

4. Test

Test the CIL model on Market-1501,

sh ./scripts/eval_market.sh

Evaluating Corruption Robustness On-the-fly

Corruption Transform

The main code of corruption transform. (See contextual code in ./datasets/make_dataloader.py, line 59)

from imagecorruptions.corruptions import *

corruption_function = [gaussian_noise, shot_noise, impulse_noise, defocus_blur,
    glass_blur, motion_blur, zoom_blur, snow, frost, fog, brightness, contrast,
    elastic_transform, pixelate, jpeg_compression, speckle_noise,
    gaussian_blur, spatter, saturate, rain]
    
class corruption_transform(object):
    def __init__(self, level=0, type='all'):
        self.level = level
        self.type = type

    def __call__(self, img):
        if self.level > 0 and self.level < 6:
            level_idx = self.level
        else:
            level_idx = random.choice(range(1, 6))
        if self.type == 'all':
            corrupt_func = random.choice(corruption_function)
        else:
            func_name_list = [f.__name__ for f in corruption_function]
            corrupt_idx = func_name_list.index(self.type)
            corrupt_func = corruption_function[corrupt_idx]
        c_img = corrupt_func(img.copy(), severity=level_idx)
        img = Image.fromarray(np.uint8(c_img))
        return img

Evaluating corruption robustness can be realized on-the-fly by modifing the transform function uesed in test dataloader. (See details in ./datasets/make_dataloader.py, Line 266)

val_with_corruption_transforms = T.Compose([
    corruption_transform(0),
    T.Resize(cfg.INPUT.SIZE_TEST),
    T.ToTensor(),])

Rain details

We introduce a rain corruption type, which is a common type of weather condition, but it is missed by the original corruption benchmark. (See details in ./datasets/make_dataloader.py, Line 27)

def rain(image, severity=1):
    if severity == 1:
        type = 'drizzle'
    elif severity == 2 or severity == 3:
        type = 'heavy'
    elif severity == 4 or severity == 5:
        type = 'torrential'
    blur_value = 2 + severity
    bright_value = -(0.05 + 0.05 * severity)
    rain = abm.Compose([
        abm.augmentations.transforms.RandomRain(rain_type=type, 
        blur_value=blur_value, brightness_coefficient=1, always_apply=True),
        abm.augmentations.transforms.RandomBrightness(limit=[bright_value, 
        bright_value], always_apply=True)])
    width, height = image.size
    if height <= 60:
        scale_factor = 65.0 / height
        new_size = (int(width * scale_factor), 65)
        image = image.resize(new_size)
    return rain(image=np.array(image))['image']

Baselines

  • Single-modality datasets
                                                                                   
Dataset Method Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
Market-1501 BoT 59.30 85.06 93.38 0.20 8.42 27.05
AGW 64.03 86.51 94.00 0.35 12.13 31.90
SBS 60.03 88.33 95.90 0.29 11.54 34.13
CIL (ours) 57.90 84.04 93.38 1.76 (0.13) 28.03 (0.45) 55.57 (0.63)
MSMT17 BoT 9.91 48.34 73.53 0.07 5.28 20.20
AGW 12.38 51.84 75.21 0.08 6.53 22.77
SBS 10.26 56.62 82.02 0.05 7.89 28.77
CIL (ours) 12.45 52.40 76.10 0.32 (0.03) 15.33 (0.20) 39.79 (0.45)
CUHK03  AGW   49.97   62.25   64.64   0.46   3.45  5.90 
 CIL (ours)   53.87   65.16   67.29   4.25 (0.39)   16.33 (0.76)   22.96 (1.04) 
  • Cross-modality datasets

Note: For RegDB dataset, Mode A and Mode B represent visible-to-thermal and thermal-to-visible experimental settings, respectively. And for SYSU-MM01 dataset, Mode A and Mode B represent all search and indoor search respectively. Note that we only corrupt RGB (visible) images in the corruption evaluation.

                                                                                                                                                                                                                                                                     
Dataset Method Mode A Mode B
Clean Eval. Corruption Eval. Clean Eval. Corruption Eval.
mINP mAP R-1 mINP mAP R-1 mINP mAP R-1 mINP mAP R-1
SYSU-MM01  AGW   36.17   47.65   47.50   14.73   29.99   34.42   59.74   62.97   54.17   35.39   40.98   33.80 
 CIL (ours)   38.15   47.64   45.51   22.48 (1.65)   35.92 (1.22)   36.95 (0.67)   57.41   60.45   50.98   43.11 (4.19)   48.65 (4.57)   40.73 (5.55) 
RegDB  AGW   54.10   68.82   75.78   32.88   43.09   45.44   52.40   68.15   75.29   6.00   41.37   67.54 
 CIL (ours)   55.68   69.75   74.96   38.66 (0.01)   49.76 (0.03)   52.25 (0.03)   55.50   69.21   74.95   11.94 (0.12)   47.90 (0.01)   67.17 (0.06)

Recent Advance in Person Re-ID

Leaderboard

Market1501-C

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
TransReID Shuting He et al. (2021) 69.29 88.93 95.07 1.98 27.38 53.19
CaceNet Fufu Yu et al. (2020) 70.47 89.82 95.40 0.67 18.24 42.92
LightMBN Fabian Herzog et al. (2021) 73.29 91.54 96.53 0.50 14.84 38.68
PLR-OS Ben Xie et al. (2020) 66.42 88.93 95.19 0.48 14.23 37.56
RRID Hyunjong Park et al. (2019) 67.14 88.43 95.19 0.46 13.45 36.57
Pyramid Feng Zheng et al. (2018) 61.61 87.50 94.86 0.36 12.75 35.72
PCB Yifan Sun et al.(2017) 41.97 82.19 94.15 0.41 12.72 34.93
BDB Zuozhuo Dai et al. (2018) 61.78 85.47 94.63 0.32 10.95 33.79
Aligned++ Hao Luo et al. (2019) 47.31 79.10 91.83 0.32 10.95 31.00
AGW Mang Ye et al. (2020) 65.40 88.10 95.00 0.30 10.80 33.40
MHN Binghui Chen et al. (2019) 55.27 85.33 94.50 0.38 10.69 33.29
LUPerson Dengpan Fu et al. (2020) 68.71 90.32 96.32 0.29 10.37 32.22
OS-Net Kaiyang Zhou et al. (2019) 56.78 85.67 94.69 0.23 10.37 30.94
VPM Yifan Sun et al. (2019) 50.09 81.43 93.79 0.31 10.15 31.17
DG-Net Zhedong Zheng et al. (2019) 61.60 86.09 94.77 0.35 9.96 31.75
ABD-Net Tianlong Chen et al. (2019) 64.72 87.94 94.98 0.26 9.81 29.65
MGN Guanshuo Wang et al.(2018) 60.86 86.51 93.88 0.29 9.72 29.56
F-LGPR Yunpeng Gong et al. (2021) 65.48 88.22 95.37 0.23 9.08 29.35
TDB Rodolfo Quispe et al. (2020) 56.41 85.77 94.30 0.20 8.90 28.56
LGPR Yunpeng Gong et al. (2021) 58.71 86.09 94.51 0.24 8.26 27.72
BoT Hao Luo et al. (2019) 51.00 83.90 94.30 0.10 6.60 26.20

CUHK03-C (detected)

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
CaceNet Fufu Yu et al. (2020) 65.22 75.13 77.64 2.09 10.62 17.04
Pyramid Feng Zheng et al. (2018) 61.41 73.14 79.54 1.10 8.03 10.42
RRID Hyunjong Park et al. (2019) 55.81 67.63 74.99 1.00 7.30 9.66
PLR-OS Ben Xie et al. (2020) 62.72 74.67 78.14 0.89 6.49 10.99
Aligned++ Hao Luo et al. (2019) 47.32 59.76 62.07 0.56 4.87 7.99
MGN Guanshuo Wang et al.(2018) 51.18 62.73 69.14 0.46 4.20 5.44
MHN Binghui Chen et al. (2019) 56.52 66.77 72.21 0.46 3.97 8.27

MSMT17-C (Version 2)

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
OS-Net Kaiyang Zhou et al. (2019) 4.05 40.05 71.86 0.08 7.86 28.51
AGW Mang Ye et al. (2020) 12.38 51.84 75.21 0.08 6.53 22.77
BoT Hao Luo et al. (2019) 9.91 48.34 73.53 0.07 5.28 20.20

Citation

Kindly include a reference to this paper in your publications if it helps your research:

@misc{chen2021benchmarks,
    title={Benchmarks for Corruption Invariant Person Re-identification},
    author={Minghui Chen and Zhiqiang Wang and Feng Zheng},
    year={2021},
    eprint={2111.00880},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
Owner
Minghui Chen
Minghui Chen
Intent parsing and slot filling in PyTorch with seq2seq + attention

PyTorch Seq2Seq Intent Parsing Reframing intent parsing as a human - machine translation task. Work in progress successor to torch-seq2seq-intent-pars

Sean Robertson 160 Jan 07, 2023
This is a repository for a Semantic Segmentation inference API using the Gluoncv CV toolkit

BMW Semantic Segmentation GPU/CPU Inference API This is a repository for a Semantic Segmentation inference API using the Gluoncv CV toolkit. The train

BMW TechOffice MUNICH 56 Nov 24, 2022
An LSTM for time-series classification

Update 10-April-2017 And now it works with Python3 and Tensorflow 1.1.0 Update 02-Jan-2017 I updated this repo. Now it works with Tensorflow 0.12. In

Rob Romijnders 391 Dec 27, 2022
Run containerized, rootless applications with podman

Why? restrict scope of file system access run any application without root privileges creates usable "Desktop applications" to integrate into your nor

119 Dec 27, 2022
Unofficial keras(tensorflow) implementation of MAE model from Masked Autoencoders Are Scalable Vision Learners

MAE-keras Unofficial keras(tensorflow) implementation of MAE model described in 'Masked Autoencoders Are Scalable Vision Learners'. This work has been

Yewon 11 Jun 12, 2022
This is the official PyTorch implementation of the paper "TransFG: A Transformer Architecture for Fine-grained Recognition" (Ju He, Jie-Neng Chen, Shuai Liu, Adam Kortylewski, Cheng Yang, Yutong Bai, Changhu Wang, Alan Yuille).

TransFG: A Transformer Architecture for Fine-grained Recognition Official PyTorch code for the paper: TransFG: A Transformer Architecture for Fine-gra

Ju He 307 Jan 03, 2023
Let Python optimize the best stop loss and take profits for your TradingView strategy.

TradingView Machine Learning TradeView is a free and open source Trading View bot written in Python. It is designed to support all major exchanges. It

Robert Roman 473 Jan 09, 2023
An AI Assistant More Than a Toolkit

tymon An AI Assistant More Than a Toolkit The reason for creating framework tymon is simple. making AI more like an assistant, helping us to complete

TymonXie 46 Oct 24, 2022
The Hailo Model Zoo includes pre-trained models and a full building and evaluation environment

Hailo Model Zoo The Hailo Model Zoo provides pre-trained models for high-performance deep learning applications. Using the Hailo Model Zoo you can mea

Hailo 50 Dec 07, 2022
PyTorch ,ONNX and TensorRT implementation of YOLOv4

PyTorch ,ONNX and TensorRT implementation of YOLOv4

4.2k Jan 01, 2023
HiFT: Hierarchical Feature Transformer for Aerial Tracking (ICCV2021)

HiFT: Hierarchical Feature Transformer for Aerial Tracking Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, and Yiming Li Our paper is Accepted by ICCV 2

Intelligent Vision for Robotics in Complex Environment 55 Nov 23, 2022
RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues

RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues FGBG (foreground-background) pytorch package for defining and training model

Klaas Kelchtermans 1 Jun 02, 2022
A computational block to solve entity alignment over textual attributes in a knowledge graph creation pipeline.

How to apply? Create your config.ini file following the example provided in config.ini Choose one of the options below to run: Run with Python3 pip in

Scientific Data Management Group 3 Jun 23, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

📖 Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) 🔥 If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 04, 2023
Matlab Python Heuristic Battery Opt - SMOP conversion and manual conversion

SMOP is Small Matlab and Octave to Python compiler. SMOP translates matlab to py

Tom Xu 1 Jan 12, 2022
AI-UPV at IberLEF-2021 EXIST task: Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models

AI-UPV at IberLEF-2021 EXIST task: Sexism Prediction in Spanish and English Tweets Using Monolingual and Multilingual BERT and Ensemble Models Descrip

Angel de Paula 1 Jun 08, 2022
Supplementary code for TISMIR paper "Sliding-Window Pitch-Class Histograms as a Means of Modeling Musical Form"

Sliding-Window Pitch-Class Histograms as a Means of Modeling Musical Form This is supplementary code for the TISMIR paper Sliding-Window Pitch-Class H

1 Nov 27, 2021
Code to reproduce the results for Statistically Robust Neural Network Classification, published in UAI 2021

Code to reproduce the results for Statistically Robust Neural Network Classification, published in UAI 2021

1 Jun 02, 2022
Alleviating Over-segmentation Errors by Detecting Action Boundaries

Alleviating Over-segmentation Errors by Detecting Action Boundaries Forked from ASRF offical code. This repo is the a implementation of replacing orig

13 Dec 12, 2022