Back to Basics: Efficient Network Compression via IMP

Back to Basics: Efficient Network Compression via IMP

Authors: Max Zimmer, Christoph Spiegel, Sebastian Pokutta

This repository contains the code to reproduce the experiments from the "Back to Basics: Efficient Network Compression via IMP" (arXiv:2111.00843) paper. The code is based on PyTorch 1.9 and the experiment-tracking platform Weights & Biases.

Structure and Usage

Experiments are started from the following three files:

• image_classification.py: Starts runs to compare pruning stable methods to IMP. IMP runs will train a model from scratch.
• distribution_image_classification.py: Starts runs to compare different sparsity allocation methods, such as Global MP or LAMP. Requires pretrained models using the Dense strategy (cf. strategies).
• iterative_image_classification.py: Starts IMP from a model that has been pretrained using the Dense strategy.

The rest of the project is structured as follows:

• strategies:
  • strategies/strategies.py: Contains all used sparsification methods.
  • strategies/iterative_strategies.py: Contains IMP that starts from a pretrained model.
• metrics: All used metrics as well as FLOP computation methods.
• models_pytorch: Contains all model architectures used.

Citation

In case you find the paper or the implementation useful for your own research, please consider citing:

@Article{zimmer2021basics,
  author        = {Max Zimmer and Christoph Spiegel and Sebastian Pokutta},
  title         = {Back to Basics: Efficient Network Compression via IMP},
  year          = {2021},
  archiveprefix = {arXiv},
  eprint        = {2111.00843},
  primaryclass  = {cs.LG},
}