Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Last update: Jun 28, 2022

Related tags

Deep Learning lottery-jackpots

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Python >= 3.7.4
Pytorch >= 1.6.1
Torchvision >= 0.4.1

Reproduce the Experiment Results

Download the pre-trained models from this link and place them in the pre-train folder.
Select a configuration file in configs to reproduce the experiment results reported in the paper. For example, to find a lottery jackpot with 30 epochs for pruning 95% parameters of ResNet-32 on CIFAR-10, run:

python cifar.py --config configs/resnet32_cifar10/90sparsity30epoch.yaml --gpus 0

To find a lottery jackpot with 30 epochs for pruning 90% parameters of ResNet-50 on ImageNet, run:

python imagenet.py --config configs/resnet50_imagenet/90sparsity30epoch.yaml --gpus 0

Note that the data_path in the yaml file should be changed to the data

Evaluate Our Pruned Models

We provide configuration, training logs, and pruned models reported in the paper. They can be downloaded from the provided links in the following table:

Model	Dataset	Sparsity	Epoch	Top-1 Acc.	Link
VGGNet-19	CIFAR-10	90%	30	93.88%	link
VGGNet-19	CIFAR-10	90%	160	93.94%	link
VGGNet-19	CIFAR-10	95%	30	93.49%	link
VGGNet-19	CIFAR-10	95%	160	93.74%	link
VGGNet-19	CIFAR-100	90%	30	72.59%	link
VGGNet-19	CIFAR-100	90%	160	74.61%	link
VGGNet-19	CIFAR-100	95%	30	71.76%	link
VGGNet-19	CIFAR-100	95%	160	73.35%	link
ResNet-32	CIFAR-10	90%	30	93.70%	link
ResNet-32	CIFAR-10	90%	160	94.39%	link
ResNet-32	CIFAR-10	95%	30	92.90%	link
ResNet-32	CIFAR-10	95%	160	93.41%	link
ResNet-32	CIFAR-100	90%	30	72.22%	link
ResNet-32	CIFAR-100	90%	160	73.43%	link
ResNet-32	CIFAR-100	95%	30	69.38%	link
ResNet-32	CIFAR-100	95%	160	70.31%	link
ResNet-50	ImageNet	80%	30	74.53%	link
ResNet-50	ImageNet	80%	60	75.26%	link
ResNet-50	ImageNet	90%	30	72.17%	link
ResNet-50	ImageNet	90%	60	72.46%	link

To test the our pruned models, download the pruned models and place them in the ckpt folder.

Select a configuration file in configs to test the pruned models. For example, to evaluate a lottery jackpot for pruning ResNet-32 on CIFAR-10, run:

python evaluate.py --config configs/resnet32_cifar10/evaluate.yaml --gpus 0

To evaluate a lottery jackpot for pruning ResNet-50 on ImageNet, run:

python evaluate.py --config configs/resnet50_imagenet/evaluate.yaml --gpus 0

Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Related tags

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Reproduce the Experiment Results

Evaluate Our Pruned Models

Owner

Yuxin Zhang

[CVPR22] Official codebase of Semantic Segmentation by Early Region Proxy.

This GitHub repo consists of Code and Some results of project- Diabetes Treatment using Gold nanoparticles. These Consist of ML Models used for prediction Diabetes and further the basic theory and working of Gold nanoparticles.

CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped

AWS documentation corpus for zero-shot open-book question answering.

Implementation of ConvMixer in TensorFlow and Keras

FS-Mol: A Few-Shot Learning Dataset of Molecules

Dynamic Divide-and-Conquer Adversarial Training for Robust Semantic Segmentation （ICCV2021）

Automated Evidence Collection for Fake News Detection

This is a virtual picture dragging application. Users may virtually slide photos across the screen. The distance between the index and middle fingers determines the movement. Smaller distances indicate click and motion, whereas bigger distances indicate only hand movement.

PyTorch implementation for the paper Pseudo Numerical Methods for Diffusion Models on Manifolds

MOT-Tracking-by-Detection-Pipeline - For Tracking-by-Detection format MOT (Multi Object Tracking), is it a framework that separates Detection and Tracking processes?

The source code for Adaptive Kernel Graph Neural Network at AAAI2022

NaturalProofs: Mathematical Theorem Proving in Natural Language

Use VITS and Opencpop to develop singing voice synthesis; Maybe it will VISinger.

An unreferenced image captioning metric (ACL-21)

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Official implementation of Meta-StyleSpeech and StyleSpeech

Official Pytorch Implementation of 3DV2021 paper: SAFA: Structure Aware Face Animation.

The authors' implementation of Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations