Code for IntraQ, PyTorch implementation of our paper under review

Last update: Nov 19, 2021

Overview

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization paper

Requirements

Python >= 3.7.10

Pytorch == 1.7.1

Reproduce results

Stage1: Generate data.

cd data_generate

Please install all required package in requirements.txt.

"--save_path_head" in run_generate_cifar10.sh/run_generate_cifar100.sh is the path where you want to save your generated data pickle.

For cifar10/100

bash run_generate_cifar10.sh
bash run_generate_cifar100.sh

For ImageNet

"--save_path_head" in run_generate.sh is the path where you want to save your generated data pickle.

"--model" in run_generate.sh is the pre-trained model you want (also is the quantized model). You can use resnet18/mobilenet_w1/mobilenetv2_w1.

bash run_generate.sh

Stage2: Train the quantized network

cd ..

Modify "qw" and "qa" in cifar10_resnet20.hocon/cifar100_resnet20.hocon/imagenet.hocon to select desired bit-width.
Modify "dataPath" in cifar10_resnet20.hocon/cifar100_resnet20.hocon/imagenet.hocon to the real dataset path (for construct the test dataloader).
Modify the "Path_to_data_pickle" in main_direct.py (line 122 and line 135) to the data_path and label_path you just generate from Stage1.
Use the below commands to train the quantized network. Please note that the model that generates the data and the quantized model should be the same.

For cifar10/100

python main_direct.py --model_name resnet20_cifar10 --conf_path cifar10_resnet20.hocon --id=0

python main_direct.py --model_name resnet20_cifar100 --conf_path cifar100_resnet20.hocon --id=0

For ImageNet, you can choose the model by modifying "--model_name" (resnet18/mobilenet_w1/mobilenetv2_w1)

python main_direct.py --model_name resnet18 --conf_path imagenet.hocon --id=0

Evaluate pre-trained models

The pre-trained models and corresponding logs can be downloaded here

Please make sure the "qw" and "qa" in *.hocon, *.hocon, "--model_name" and "--model_path" are correct.

For cifar10/100

python test.py --model_name resnet20_cifar10 --model_path path_to_pre-trained model --conf_path cifar10_resnet20.hocon

python test.py --model_name resnet20_cifar100 --model_path path_to_pre-trained model --conf_path cifar100_resnet20.hocon

For ImageNet

python test.py --model_name resnet18/mobilenet_w1/mobilenetv2_w1 --model_path path_to_pre-trained model --conf_path imagenet.hocon

Results of pre-trained models are shown below:

Model	Bit-width	Dataset	Top-1 Acc.
resnet18	W4A4	ImageNet	66.47%
resnet18	W5A5	ImageNet	69.94%
mobilenetv1	W4A4	ImageNet	51.36%
mobilenetv1	W5A5	ImageNet	68.17%
mobilenetv2	W4A4	ImageNet	65.10%
mobilenetv2	W5A5	ImageNet	71.28%
resnet-20	W3A3	cifar10	77.07%
resnet-20	W4A4	cifar10	91.49%
resnet-20	W3A3	cifar100	64.98%
resnet-20	W4A4	cifar100	48.25%

Code for IntraQ, PyTorch implementation of our paper under review

Related tags

Overview

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization paper

Requirements

Reproduce results

Stage1: Generate data.

Stage2: Train the quantized network

Evaluate pre-trained models

Owner

A framework for multi-step probabilistic time-series/demand forecasting models

An implementation of RetinaNet in PyTorch.

ANEA: Distant Supervision for Low-Resource Named Entity Recognition

Spectral Temporal Graph Neural Network (StemGNN in short) for Multivariate Time-series Forecasting

Anchor-free Oriented Proposal Generator for Object Detection

HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty

A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.

LSTMs (Long Short Term Memory) RNN for prediction of price trends

PyTorch implementation for ACL 2021 paper "Maria: A Visual Experience Powered Conversational Agent".

GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training @ KDD 2020

Addon and nodes for working with structural biology and molecular data in Blender.

Video Frame Interpolation without Temporal Priors (a general method for blurry video interpolation)

Inferring Lexicographically-Ordered Rewards from Preferences

This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

The code for Expectation-Maximization Attention Networks for Semantic Segmentation (ICCV'2019 Oral)

Official PyTorch implementation of "Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning" (ICCV2021 Oral)

Learning to Communicate with Deep Multi-Agent Reinforcement Learning in PyTorch

Arbitrary Distribution Modeling with Censorship in Real Time 59 2 60 3 Bidding Advertising for KDD'21

Code for generating the figures in the paper "Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?"

Kalman Filter book using Jupyter Notebook. Focuses on building intuition and experience, not formal proofs. Includes Kalman filters,extended Kalman filters, unscented Kalman filters, particle filters, and more. All exercises include solutions.