Awesome Pruning

A curated list of neural network pruning and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers and Awesome-NAS.

Please feel free to pull requests or open an issue to add papers.

Type of Pruning
2020 Venues
2019 Venues
2018 Venues
2017 Venues
2016 Venues
2015 Venues

Type of Pruning

Type	`F`	`W`	`Other`
Explanation	Filter pruning	Weight pruning	other types

2020

Title	Venue	Type	Code
HYDRA: Pruning Adversarially Robust Neural Networks	NeurIPS	`W`	PyTorch(Author)
Logarithmic Pruning is All You Need	NeurIPS	`W`	-
Directional Pruning of Deep Neural Networks	NeurIPS	`W`	-
Movement Pruning: Adaptive Sparsity by Fine-Tuning	NeurIPS	`W`	PyTorch(Author)
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot	NeurIPS	`W`	PyTorch(Author)
Neuron Merging: Compensating for Pruned Neurons	NeurIPS	`F`	PyTorch(Author)
Neuron-level Structured Pruning using Polarization Regularizer	NeurIPS	`F`	PyTorch(Author)
SCOP: Scientific Control for Reliable Neural Network Pruning	NeurIPS	`F`	-
Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning	NeurIPS	`F`	-
The Generalization-Stability Tradeoff In Neural Network Pruning	NeurIPS	`F`	PyTorch(Author)
Pruning Filter in Filter	NeurIPS	`Other`	PyTorch(Author)
Position-based Scaled Gradient for Model Quantization and Pruning	NeurIPS	`Other`	PyTorch(Author)
Bayesian Bits: Unifying Quantization and Pruning	NeurIPS	`Other`	-
Pruning neural networks without any data by iteratively conserving synaptic flow	NeurIPS	`Other`	PyTorch(Author)
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning	ECCV (Oral)	`F`	PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation	ECCV	`F`	-
DHP: Differentiable Meta Pruning via HyperNetworks	ECCV	`F`	PyTorch(Author)
Meta-Learning with Network Pruning	ECCV	`W`	-
Accelerating CNN Training by Pruning Activation Gradients	ECCV	`W`	-
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search	ECCV	`Other`	-
Differentiable Joint Pruning and Quantization for Hardware Efficiency	ECCV	`Other`	-
Channel Pruning via Automatic Structure Search	IJCAI	`F`	PyTorch(Author)
Adversarial Neural Pruning with Latent Vulnerability Suppression	ICML	`W`	-
Proving the Lottery Ticket Hypothesis: Pruning is All You Need	ICML	`W`	-
Soft Threshold Weight Reparameterization for Learnable Sparsity	ICML	`WF`	Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection	ICML	`F`	-
Operation-Aware Soft Channel Pruning using Differentiable Masks	ICML	`F`	-
DropNet: Reducing Neural Network Complexity via Iterative Pruning	ICML	`F`	-
Towards Efficient Model Compression via Learned Global Ranking	CVPR (Oral)	`F`	Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map	CVPR (Oral)	`F`	Pytorch(Author)
Neural Network Pruning with Residual-Connections and Limited-Data	CVPR (Oral)	`F`	-
Multi-Dimensional Pruning: A Unified Framework for Model Compression	CVPR (Oral)	`WF`	-
DMCP: Differentiable Markov Channel Pruning for Neural Networks	CVPR (Oral)	`F`	TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression	CVPR	`F`	PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression	CVPR	`F`	-
Discrete Model Compression With Resource Constraint for Deep Neural Networks	CVPR	`F`	-
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization	CVPR	`W`	-
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration	CVPR	`F`	-
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy	CVPR	`F`	-
Comparing Rewinding and Fine-tuning in Neural Network Pruning	ICLR (Oral)	`WF`	TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization	ICLR (Spotlight)	`W`	-
ProxSGD: Training Structured Neural Networks under Regularization and Constraints	ICLR	`W`	TF+PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation	ICLR	`W`	-
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning	ICLR	`W`	PyTorch(Author)
Dynamic Model Pruning with Feedback	ICLR	`WF`	-
Provable Filter Pruning for Efficient Neural Networks	ICLR	`F`	-
Data-Independent Neural Pruning via Coresets	ICLR	`W`	-
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates	AAAI	`F`	-
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks	AAAI	`Other`	-
Pruning from Scratch	AAAI	`Other`	-

2019

Title	Venue	Type	Code
Network Pruning via Transformable Architecture Search	NeurIPS	`F`	PyTorch(Author)
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks	NeurIPS	`F`	PyTorch(Author)
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask	NeurIPS	`W`	TensorFlow(Author)
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers	NeurIPS	`W`	-
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks	NeurIPS	`W`	PyTorch(Author)
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters	NeurIPS	`W`	-
Model Compression with Adversarial Robustness: A Unified Optimization Framework	NeurIPS	`Other`	PyTorch(Author)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning	ICCV	`F`	PyTorch(Author)
Accelerate CNN via Recursive Bayesian Pruning	ICCV	`F`	-
Adversarial Robustness vs Model Compression, or Both?	ICCV	`W`	PyTorch(Author)
Learning Filter Basis for Convolutional Neural Network Compression	ICCV	`Other`	-
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration	CVPR (Oral)	`F`	PyTorch(Author)
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning	CVPR	`F`	PyTorch(Author)
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure	CVPR	`F`	PyTorch(Author)
On Implicit Filter Level Sparsity in Convolutional Neural Networks, Extension1, Extension2	CVPR	`F`	PyTorch(Author)
Structured Pruning of Neural Networks with Budget-Aware Regularization	CVPR	`F`	-
Importance Estimation for Neural Network Pruning	CVPR	`F`	PyTorch(Author)
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks	CVPR	`F`	-
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search	CVPR	`Other`	TensorFlow(Author)
Variational Convolutional Neural Network Pruning	CVPR	-	-
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks	ICLR (Best)	`W`	TensorFlow(Author)
Rethinking the Value of Network Pruning	ICLR	`F`	PyTorch(Author)
Dynamic Channel Pruning: Feature Boosting and Suppression	ICLR	`F`	TensorFlow(Author)
SNIP: Single-shot Network Pruning based on Connection Sensitivity	ICLR	`W`	TensorFLow(Author)
Dynamic Sparse Graph for Efficient Deep Learning	ICLR	`F`	CUDA(3rd)
Collaborative Channel Pruning for Deep Networks	ICML	`F`	-
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github	ICML	`F`	-
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis4	ICML	`W`	PyTorch(Author)
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning	IJCAI	`F`	Tensorflow(Author)

2018

Title	Venue	Type	Code
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers	ICLR	`F`	TensorFlow(Author), PyTorch(3rd)
To prune, or not to prune: exploring the efficacy of pruning for model compression	ICLR	`W`	-
Discrimination-aware Channel Pruning for Deep Neural Networks	NeurIPS	`F`	TensorFlow(Author)
Frequency-Domain Dynamic Pruning for Convolutional Neural Networks	NeurIPS	`W`	-
Learning Sparse Neural Networks via Sensitivity-Driven Regularization	NeurIPS	`WF`	-
Amc: Automl for model compression and acceleration on mobile devices	ECCV	`F`	TensorFlow(3rd)
Data-Driven Sparse Structure Selection for Deep Neural Networks	ECCV	`F`	MXNet(Author)
Coreset-Based Neural Network Compression	ECCV	`F`	PyTorch(Author)
Constraint-Aware Deep Neural Network Compression	ECCV	`W`	SkimCaffe(Author)
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers	ECCV	`W`	Caffe(Author)
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning	CVPR	`F`	PyTorch(Author)
NISP: Pruning Networks using Neuron Importance Score Propagation	CVPR	`F`	-
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization	CVPR	`W`	-
“Learning-Compression” Algorithms for Neural Net Pruning	CVPR	`W`	-
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks	IJCAI	`F`	PyTorch(Author)
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning	IJCAI	`F`	-

2017

Title	Venue	Type	Code
Pruning Filters for Efficient ConvNets	ICLR	`F`	PyTorch(3rd)
Pruning Convolutional Neural Networks for Resource Efficient Inference	ICLR	`F`	TensorFlow(3rd)
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee	NeurIPS	`W`	TensorFlow(Author)
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon	NeurIPS	`W`	PyTorch(Author)
Runtime Neural Pruning	NeurIPS	`F`	-
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning	CVPR	`F`	-
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression	ICCV	`F`	Caffe(Author), PyTorch(3rd)
Channel pruning for accelerating very deep neural networks	ICCV	`F`	Caffe(Author)
Learning Efficient Convolutional Networks Through Network Slimming	ICCV	`F`	PyTorch(Author)

2016

Title	Venue	Type	Code
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding	ICLR (Best)	`W`	Caffe(Author)
Dynamic Network Surgery for Efficient DNNs	NeurIPS	`W`	Caffe(Author)

2015

Title	Venue	Type	Code
Learning both Weights and Connections for Efficient Neural Networks	NeurIPS	`W`	PyTorch(3rd)

EfficientDNNs

Embedded-Neural-Network

awesome-AutoML-and-Lightweight-Models

Model-Compression-Papers

knowledge-distillation-papers

Network-Speed-and-Compression

A curated list of neural network pruning resources.

Related tags

Overview

Awesome Pruning

Table of Contents

Type of Pruning

2020

2019

2018

2017

2016

2015

Related Repo

Owner

Yang He

My implementation of DeepMind's Perceiver

Paper list of log-based anomaly detection

Yolo Traffic Light Detection With Python

PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems

I will implement Fastai in each projects present in this repository.

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

Pervasive Attention: 2D Convolutional Networks for Sequence-to-Sequence Prediction

Image augmentation library in Python for machine learning.

Flower - A Friendly Federated Learning Framework

Applying PVT to Semantic Segmentation

[CVPR 2021] Teachers Do More Than Teach: Compressing Image-to-Image Models (CAT)

A flexible submap-based framework towards spatio-temporally consistent volumetric mapping and scene understanding.

This is the official PyTorch implementation for "Mesa: A Memory-saving Training Framework for Transformers".

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

Hierarchical Metadata-Aware Document Categorization under Weak Supervision (WSDM'21)

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

An official source code for "Augmentation-Free Self-Supervised Learning on Graphs"

Deep Learning for humans

Re-implementation of 'Grokking: Generalization beyond overfitting on small algorithmic datasets'

Code for "Long Range Probabilistic Forecasting in Time-Series using High Order Statistics"