Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

Related tags

Deep LearningDisAlign
Overview

Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

[Paper][Code]

We implement the classification, object detection and instance segmentation tasks based on our cvpods. The users should install cvpods first and run the experiments in this repo.

Changelog

  • 4.23.2021 Update the DisAlign on LVIS v0.5(Mask R-CNN + Res50)
  • 4.12.2021 Update the README

0. How to Use

  • Step-1: Install the latest cvpods.
  • Step-2: cd cvpods
  • Step-3: Prepare dataset for different tasks.
  • Step-4: git clone https://github.com/Megvii-BaseDetection/DisAlign playground_disalign
  • Step-5: Enter one folder and run pods_train --num-gpus 8
  • Step-6: Use pods_test --num-gpus 8 to evaluate the last the checkpoint

1. Image Classification

We support the the following three datasets:

  • ImageNet-LT Dataset
  • iNaturalist-2018 Dataset
  • Place-LT Dataset

We refer the user to CLS_README for more details.

2. Object Detection/Instance Segmentation

We support the two versions of the LVIS dataset:

  • LVIS v0.5
  • LVIS v1.0

Highlight

  1. To speedup the evaluation on LVIS dataset, we provide the C++ optimized evaluation api by modifying the coco_eval(C++) in cvpods.
  • The C++ version lvis_eval API will save ~30% time when calculating the mAP.
  1. We provide support for the metric of AP_fixed and AP_pool proposed in large-vocab-devil
  2. We will support more recent works on long-tail detection in this project(e.g. EQLv2, CenterNet2, etc.) in the future.

We refer the user to DET_README for more details.

3. Semantic Segmentation

We adopt the mmsegmentation as the codebase for runing all experiments of DisAlign. Currently, the user should use DisAlign_Seg for the semantic segmentation experiments. We will add the support for these experiments in cvpods in the future.

Acknowledgement

Thanks for the following projects:

Citing DisAlign

If you are using the DisAlign in your research or with to refer to the baseline results publised in this repo, please use the following BibTex entry.

@inproceedings{zhang2021disalign,
  title={Distribution Alignment: A Unified Framework for Long-tail Visual Recognition.},
  author={Zhang, Songyang and Li, Zeming and Yan, Shipeng and He, Xuming and Sun, Jian},
  booktitle={CVPR},
  year={2021}
}

License

This repo is released under the Apache 2.0 license. Please see the LICENSE file for more information.

Comments
  • scale in cosine classifier

    scale in cosine classifier

    Hi, thanks for your great work! I notice you use the cosine classifier in many experiments and it can get a better baseline. The formula is as follows

    image

    I am wondering the value of s?

    opened by L1aoXingyu 5
  •  Is it correct to freeze the weight and bias of the DisAlign Linear Layer as well?

    Is it correct to freeze the weight and bias of the DisAlign Linear Layer as well?

    Hello. Thank you for your project! I'm testing your code on my custom dataset. My task is classification. I have a question about your code implementation.

    https://github.com/Megvii-BaseDetection/DisAlign/blob/a2fc3500a108cb83e3942293a5675c97ab3a2c6e/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/net.py#L56-L62

    From my understanding, in stage 2, remove the linear layer used in stage 1 and add DisAlign Linear Layer. And freeze all parts except for logit_scale, logit_bias, and confidence_layer. At this time, the weight and bias of DisAlignLinear are also frozen. (self.weight, self.bias) Is my understanding correct?

    If so, are the weight and bias of DisAlignLinearLayer fixed after the initialization? (The weight and bias of the linear layer in stage 1 are not copied either)

    If my understanding is correct, why is the weight of DisAlignLinear also frozen?

    I will wait for your reply. thanks!

    opened by jeongHwarr 4
  • Where is the DisAlignLinear module?

    Where is the DisAlignLinear module?

    Hello. Thank you for your impressive project!

    I want to apply DisAlign to classification. However, an error occurs in the import part. https://github.com/Megvii-BaseDetection/DisAlign/blob/a2fc3500a108cb83e3942293a5675c97ab3a2c6e/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/net.py#L7 I coudn't find the DisAlignLinear in cvpods.layers. and there also isn't exist at https://github.com/Megvii-BaseDetection/cvpods/tree/master/cvpods/layers How can I solve this problem?

    Thank you!

    opened by jeongHwarr 4
  • Can someone kindly share their codes of Classification task on ImageNet_LT?

    Can someone kindly share their codes of Classification task on ImageNet_LT?

    I tried to train the proposed method on ImageNet_LT, but I can only get an average testing rate about 49%, which is far from the rate described in the paper (52.9). Some of the details regarding my implementations are given as follows: (1) The feature extractor is ResNexT-50 and the head classifier is a linear classifier. The testing accuracy in Stage-One is 43.9%, which is OK.

    (2) The testing accuracy of adopting cRT method in Stage-Two is 49.6%, which is identical to one reported in other papers. (3) When fine-tuning the model in Stage-2, both the feature-extractor and head classifier are frozen, and a DisAliLinear model (which is implemented in CVPODs) is retrained. The testing accuracy can only reach 48.8%, which is far away from the one reported in your paper.

    opened by smallcube 4
  • The code for semantic segmentation is missing

    The code for semantic segmentation is missing

    Hi, thank you for the nice work, but the code for semantic segmentation is missing and the URL for it in the README could not be opened. Could you please fix this issue?

    opened by curiosity654 3
  • About the reference Distribution p_r in Eq. (10)

    About the reference Distribution p_r in Eq. (10)

    Hi, Thank you for providing your code. Here I was wondering the Equation (10) in your paper (The definition of p_r), which seems not to be a distribution. Since every x_i can only have one label, the reference distribution p_r(y| x_i) will be the distribution like (0, 0, 0,...,w_c, 0, 0,...,0). And the sum of this distribution is w_c, but not 1.

    Could you help me understand this equation? Thanks in advance.

    opened by Kevinz-code 3
  • import error

    import error

    Hi, thanks for the great work. Maybe I missed it, but it seems that the code for this project has been incorporated into cvpods. I couldn't launch any experiments due to ImportErrors like: from cvpods.layers import DisAlignLinear ImportError: cannot import name 'DisAlignLinear' from 'cvpods.layers' Also, I didn't find the corresponding functions in cvpods.

    Any help will be appreciated. Thanks.

    opened by YUE-FAN 2
  • about the confidence score σ(x)

    about the confidence score σ(x)

    In the paper, the σ(x) is implemented as a linear layer followed by a non-linear activation function (e.g., sigmoid function) for all input x. How to understand the input x?the matrix of raw iamge, or the extracted features, even or cls_score? Thank you!

    opened by lzed2399 2
  • exp_reweight = exp_reweight / np.sum(exp_reweight) * num_foreground

    exp_reweight = exp_reweight / np.sum(exp_reweight) * num_foreground

    Dear author, I have some questions about the code and paper:

    1. exp_reweight = exp_reweight / np.sum(exp_reweight) * num_foreground Why "exp_reweight" is multiplied by the coefficient "num_foreground"? It is not mentioned in the paper.
    2. Is "K" in the empirical class frequencies r = [r1, · · · , rK] on the training set in the paper the same as the class number C of the training set?
    opened by Liu-wanbing 2
  • The DisAlign_Seg page can't open

    The DisAlign_Seg page can't open

    opened by Kittywyk 1
  • Do you use validation dataset?

    Do you use validation dataset?

    https://github.com/Megvii-BaseDetection/DisAlign/blob/main/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/config.py#L31

    It seems that you only use test dataset? What is the reason for that?

    opened by qianlanwyd 1
  • How can I test and augtest the trained semseg DisAlign model?

    How can I test and augtest the trained semseg DisAlign model?

    opened by jh151170 0
  • the code question in semantic_seg

    the code question in semantic_seg

    Hi, I have a questation about the logit_scale and logit_bias in semantic_seg. The shape of the above parameter is (1, num_classes, 1, 1), why not is (1, num_classes, 512, 512) which is matched the input image size for semantic segmenation.

    opened by Ianresearch 8
  • Value of the learned scale and bias vector?

    Value of the learned scale and bias vector?

    Hi, did you check the value change of the learned scale and bias vector throughout the training process? I find the value of them change in the first few iterations and remain stable in the rest time on my own classification dataset. I wonder how the learned vectors look like in your paper? Thanks!

    opened by Jacobew 1
Owner
BaseDetection Team of Megvii
[3DV 2021] Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation

Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation This is the official implementation for the method described in Ch

Jiaxing Yan 27 Dec 30, 2022
Top #1 Submission code for the first https://alphamev.ai MEV competition with best AUC (0.9893) and MSE (0.0982).

alphamev-winning-submission Top #1 Submission code for the first alphamev MEV competition with best AUC (0.9893) and MSE (0.0982). The code won't run

70 Oct 29, 2022
Code Repository for The Kaggle Book, Published by Packt Publishing

The Kaggle Book Data analysis and machine learning for competitive data science Code Repository for The Kaggle Book, Published by Packt Publishing "Lu

Packt 1.6k Jan 07, 2023
Blender Add-on that sets a Material's Base Color to one of Pantone's Colors of the Year

Blender PCOY (Pantone Color of the Year) MCMC (Mid-Century Modern Colors) HG71 (House & Garden Colors 1971) Blender Add-ons That Assign a Custom Color

Don Schnitzius 15 Nov 20, 2022
NFNets and Adaptive Gradient Clipping for SGD implemented in PyTorch

PyTorch implementation of Normalizer-Free Networks and SGD - Adaptive Gradient Clipping Paper: https://arxiv.org/abs/2102.06171.pdf Original code: htt

Vaibhav Balloli 320 Jan 02, 2023
D²Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos

D²Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos This repository contains the implementation for "D²Conv3D: Dynamic Dilated Co

17 Oct 20, 2022
The fastai book, published as Jupyter Notebooks

English / Spanish / Korean / Chinese / Bengali / Indonesian The fastai book These notebooks cover an introduction to deep learning, fastai, and PyTorc

fast.ai 17k Jan 07, 2023
[AAAI 2022] Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification

Sparse Structure Learning via Graph Neural Networks for inductive document classification Make graph dataset create co-occurrence graph for datasets.

16 Dec 22, 2022
This repository contains the code for our fast polygonal building extraction from overhead images pipeline.

Polygonal Building Segmentation by Frame Field Learning We add a frame field output to an image segmentation neural network to improve segmentation qu

Nicolas Girard 186 Jan 04, 2023
Gluon CV Toolkit

Gluon CV Toolkit | Installation | Documentation | Tutorials | GluonCV provides implementations of the state-of-the-art (SOTA) deep learning models in

Distributed (Deep) Machine Learning Community 5.4k Jan 06, 2023
Implement face detection, and age and gender classification, and emotion classification.

YOLO Keras Face Detection Implement Face detection, and Age and Gender Classification, and Emotion Classification. (image from wider face dataset) Ove

Chloe 10 Nov 14, 2022
Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 short.

Session-aware BERT4Rec Official repository for "Exploiting Session Information in BERT-based Session-aware Sequential Recommendation", SIGIR 2022 shor

Jamie J. Seol 22 Dec 13, 2022
ConvMAE: Masked Convolution Meets Masked Autoencoders

ConvMAE ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao1, Teli Ma1, Hongsheng Li2, Jifeng Dai3, Yu Qiao1, 1 Shanghai AI Laboratory, 2 M

Alpha VL Team of Shanghai AI Lab 345 Jan 08, 2023
Unofficial Tensorflow 2 implementation of the paper Implicit Neural Representations with Periodic Activation Functions

Siren: Implicit Neural Representations with Periodic Activation Functions The unofficial Tensorflow 2 implementation of the paper Implicit Neural Repr

Seyma Yucer 2 Jun 27, 2022
A PoC Corporation Relationship Knowledge Graph System on top of Nebula Graph.

Corp-Rel is a PoC of Corpartion Relationship Knowledge Graph System. It's built on top of the Open Source Graph Database: Nebula Graph with a dataset

Wey Gu 20 Dec 11, 2022
Auto-Encoding Score Distribution Regression for Action Quality Assessment

DAE-AQA It is an open source program reference to paper Auto-Encoding Score Distribution Regression for Action Quality Assessment. 1.Introduction DAE

13 Nov 16, 2022
RL-driven agent playing tic-tac-toe on starknet against challengers.

tictactoe-on-starknet RL-driven agent playing tic-tac-toe on starknet against challengers. GUI reference: https://pythonguides.com/create-a-game-using

21 Jul 30, 2022
[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

Stable Head Pose Estimation and Landmark Regression via 3D Dense Face Reconstruction Reimplementation of (ECCV 2020) Towards Fast, Accurate and Stable

Remilia Scarlet 221 Dec 30, 2022
Nvidia Semantic Segmentation monorepo

Paper | YouTube | Cityscapes Score Pytorch implementation of our paper Hierarchical Multi-Scale Attention for Semantic Segmentation. Please refer to t

NVIDIA Corporation 1.6k Jan 04, 2023
Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations Trevor Ablett, Daniel (Yifan) Zhai, Jonatha

STARS Laboratory 3 Feb 01, 2022