Semi-Supervised Learning for Fine-Grained Classification

Overview

Semi-Supervised Learning for Fine-Grained Classification

This repo contains the code of:

  • A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification, Jong-Chyi Su, Zezhou Cheng, and Subhransu Maji, CVPR 2021. [paper, poster, slides]
  • Semi-Supervised Learning with Taxonomic Labels, Jong-Chyi Su and Subhransu Maji, BMVC 2021. [paper, slides]

Preparing Datasets and Splits

We used the following datasets in the paper:

In addition the repository contains a new Semi-iNat dataset corresponding to the FGVC8 semi-supervised challenge:

  • Semi-iNat: This is a new dataset for the Semi-iNat Challenge at FGVC8 workshop at CVPR 2021. Different from Semi-Aves, Semi-iNat has more species from different kingdoms, and does not include in or out-of-domain label. For more details please see the challenge website.

The splits of each of these datasets can be found under data/${dataset}/${split}.txt corresponding to:

  • l_train -- labeled in-domain data
  • u_train_in -- unlabeled in-domain data
  • u_train_out -- unlabeled out-of-domain data
  • u_train (combines u_train_in and u_train_out)
  • val -- validation set
  • l_train_val (combines l_train and val)
  • test -- test set

Each line in the text file has a filename and the corresponding class label.

Please download the datasets from the corresponding websites. For Semi-Aves, put the data under data/semi_aves. FFor Semi-Fungi and Semi-CUB, download the images and put them under data/semi_fungi/images and data/cub/images.

Note 1: For the experiments on Semi-Fungi reported in the paper, the images are resized to a maximum of 300px for each side.
Note 2: We reported the results of another split of Semi-Aves in the appendix (for cross-validation), but we do not release the labels because it will leak the labels for unlabeled data.
Note 3: We also provide the species names of Semi-Aves under data/semi_aves_species_names.txt, and the species names of Semi-Fungi. The names were not shared in the competetion.

Training and Evaluation (CVPR paper)

We provide the code for all the methods included in the paper, except for FixMatch and MoCo. This includes methods of supervised training, self-training, PL, and curriculum PL. This code is developed based on this PyTorch implementation.

For FixMatch, we used the official Tensorflow code and an unofficial PyTorch code to reproduce the results. For MoCo, we use this PyContrast implementation.

To train the model, use the following command:

CUDA_VISIBLE_DEVICES=0 python run_train.py --task ${task} --init ${init} --alg ${alg} --unlabel ${unlabel} --num_iter ${num_iter} --warmup ${warmup} --lr ${lr} --wd ${wd} --batch_size ${batch_size} --exp_dir ${exp_dir} --MoCo ${MoCo} --alpha ${alpha} --kd_T ${kd_T} --trainval

For example, to train a supervised model initialized from a inat pre-trained model on semi-aves dataset with in-domain unlabeled data only, you will use:

CUDA_VISIBLE_DEVICES=0 python run_train.py --task semi_aves --init inat --alg supervised --unlabel in --num_iter 10000 --lr 1e-3 --wd 1e-4 --exp_dir semi_aves_supervised_in --MoCo false --trainval

Note that for experiments of Semi-Aves and Semi-Fungi in the paper, we combined the training and val set for training (use args --trainval).
For all the hyper-parameters, please see the following shell scripts:

  • exp_sup.sh for supervised training
  • exp_PL.sh for pseudo-labeling
  • exp_CPL.sh for curriculum pseudo-labeling
  • exp_MoCo.sh for MoCo + supervised training
  • exp_distill.sh for self-training and MoCo + self-training

Training and Evaluation (BMVC paper)

In our BMVC paper, we added the hierarchical supervision of coarse labels on top of semi-supervised learning.

To train the model, use the following command:

CUDA_VISIBLE_DEVICES=0 python run_train_hierarchy.py --task ${task} --init ${init} --alg ${alg} --unlabel ${unlabel} --num_iter ${num_iter} --warmup ${warmup} --lr ${lr} --wd ${wd} --batch_size ${batch_size} --exp_dir ${exp_dir} --MoCo ${MoCo} --alpha ${alpha} --kd_T ${kd_T} --level ${level}

The following are the arguments different from the above:

  • ${level}: choose from {genus, kingdom, phylum, class, order, family, species}
  • ${alg}: choose from {hierarchy, PL_hierarchy, distill_hierarchy}

For the settings and hyper-parameters, please see exp_hierarchy.sh.

Pre-Trained Models

We provide supervised training models, MoCo pre-trained models, as well as MoCo + supervised training models, for both Semi-Aves and Semi-Fungi datasets. Here are the links to download the model:

http://vis-www.cs.umass.edu/semi-inat-2021/ssl_evaluation/models/${method}/${dataset}_${initialization}_${unlabel}.pth.tar

  • ${method}: choose from {supervised, MoCo_init, MoCo_supervised}
  • ${dataset}: choose from {semi_aves, semi_fungi}
  • ${initialization}: choose from {scratch, imagenet, inat}
  • ${unlabel}: choose from {in, inout}

You need these models for self-training mothods. For example, the teacher model is initialized from model/supervised for self-training. For MoCo + self-training, the teacher model is initialized from model/MoCo_supervised, and the student model is initialized from model/MoCo_init.

We also provide the pre-trained ResNet-50 model of iNaturalist-18. This model was trained using this github code.

Related Challenges

Citation

@inproceedings{su2021realistic,
  author    = {Jong{-}Chyi Su and Zezhou Cheng and Subhransu Maji},
  title     = {A Realistic Evaluation of Semi-Supervised Learning for Fine-Grained Classification},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2021}
}

@inproceedings{su2021taxonomic,
  author    = {Jong{-}Chyi Su and Subhransu Maji},
  title     = {Semi-Supervised Learning with Taxonomic Labels},
  booktitle = {British Machine Vision Conference (BMVC)},
  year      = {2021}
}

@article{su2021semi_iNat,
      title={The Semi-Supervised iNaturalist Challenge at the FGVC8 Workshop}, 
      author={Jong-Chyi Su and Subhransu Maji},
      year={2021},
      journal={arXiv preprint arXiv:2106.01364}
}

@article{su2021semi_aves,
      title={The Semi-Supervised iNaturalist-Aves Challenge at FGVC7 Workshop}, 
      author={Jong-Chyi Su and Subhransu Maji},
      year={2021},
      journal={arXiv preprint arXiv:2103.06937}
}
Simple-System-Convert--C--F - Simple System Convert With Python

Simple-System-Convert--C--F REQUIREMENTS Python version : 3 HOW TO USE Run the c

Jonathan Santos 2 Feb 16, 2022
Data and code from COVID-19 machine learning paper

Machine learning approaches for localized lockdown, subnotification analysis and cases forecasting in São Paulo state counties during COVID-19 pandemi

Sara Malvar 4 Dec 22, 2022
Implementation of Diverse Semantic Image Synthesis via Probability Distribution Modeling

Diverse Semantic Image Synthesis via Probability Distribution Modeling (CVPR 2021) Paper Zhentao Tan, Menglei Chai, Dongdong Chen, Jing Liao, Qi Chu,

tzt 45 Nov 17, 2022
Full body anonymization - Realistic Full-Body Anonymization with Surface-Guided GANs

Realistic Full-Body Anonymization with Surface-Guided GANs This is the official

Håkon Hukkelås 30 Nov 18, 2022
Code for Active Learning at The ImageNet Scale.

Code for Active Learning at The ImageNet Scale. This repository implements many popular active learning algorithms and allows training with torch's DDP.

Zeyad Emam 47 Dec 12, 2022
A web application that provides real time temperature and humidity readings of a house.

About A web application which provides real time temperature and humidity readings of a house. If you're interested in the data collected so far click

Ben Thompson 3 Jan 28, 2022
DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

Microsoft 8.4k Jan 01, 2023
ECAENet (TensorFlow and Keras)

ECAENet: EfficientNet with Efficient Channel Attention for Plant Species Recognition (SCI:Q3) (Journal of Intelligent & Fuzzy Systems)

4 Dec 22, 2022
NeurIPS 2021, self-supervised 6D pose on category level

SE(3)-eSCOPE video | paper | website Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation Xiaolong Li, Yijia Weng,

Xiaolong 63 Nov 22, 2022
Match SafeGraph POIs with Data collected through a cultural resource survey in Washington DC.

Match SafeGraph POI data with Cultural Resource Places in Washington DC Match SafeGraph POIs with Data collected through a cultural resource survey in

Changjie Chen 1 Jan 05, 2022
Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.

CycleGAN PyTorch | project page | paper Torch implementation for learning an image-to-image translation (i.e. pix2pix) without input-output pairs, for

Jun-Yan Zhu 11.5k Dec 30, 2022
A PyTorch implementation of "Graph Classification Using Structural Attention" (KDD 2018).

GAM ⠀⠀ A PyTorch implementation of Graph Classification Using Structural Attention (KDD 2018). Abstract Graph classification is a problem with practic

Benedek Rozemberczki 259 Dec 05, 2022
Diagnostic tests for linguistic capacities in language models

LM diagnostics This repository contains the diagnostic datasets and experimental code for What BERT is not: Lessons from a new suite of psycholinguist

61 Jan 02, 2023
a delightful machine learning tool that allows you to train, test and use models without writing code

igel A delightful machine learning tool that allows you to train/fit, test and use models without writing code Note I'm also working on a GUI desktop

Nidhal Baccouri 3k Jan 05, 2023
A PoC Corporation Relationship Knowledge Graph System on top of Nebula Graph.

Corp-Rel is a PoC of Corpartion Relationship Knowledge Graph System. It's built on top of the Open Source Graph Database: Nebula Graph with a dataset

Wey Gu 20 Dec 11, 2022
[ICCV'21] UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction Project Page | Paper | Supplementary | Video This reposit

331 Dec 28, 2022
NeuralDiff: Segmenting 3D objects that move in egocentric videos

NeuralDiff: Segmenting 3D objects that move in egocentric videos Project Page | Paper + Supplementary | Video About This repository contains the offic

Vadim Tschernezki 14 Dec 05, 2022
Test-Time Personalization with a Transformer for Human Pose Estimation, NeurIPS 2021

Transforming Self-Supervision in Test Time for Personalizing Human Pose Estimation This is an official implementation of the NeurIPS 2021 paper: Trans

41 Nov 28, 2022
git《Commonsense Knowledge Base Completion with Structural and Semantic Context》(AAAI 2020) GitHub: [fig1]

Commonsense Knowledge Base Completion with Structural and Semantic Context Code for the paper Commonsense Knowledge Base Completion with Structural an

AI2 96 Nov 05, 2022
Implement of "Training deep neural networks via direct loss minimization" in PyTorch for 0-1 loss

This is the implementation of "Training deep neural networks via direct loss minimization" published at ICML 2016 in PyTorch. The implementation targe

Cuong Nguyen 1 Jan 18, 2022