DA2Lite is an automated model compression toolkit for PyTorch.

DA2Lite (Deep Architecture to Lite) is a toolkit to compress and accelerate deep network models.

⭐ Star us on GitHub — it helps!!

	Frameworks & Libraries	Algorithms
Built-in	Supported Framework PyTorch Examples resnet18_tucker	Pruning L1 norm pruning L2 norm pruning EagleEye Slimming Filter Decomposition CP decomposition Tucker decomposition VBMF for rank selection Knowledge Distillation FSKD

Install

git clone https://github.com/da2so/DA2Lite.git

You will need a machine with a GPU and CUDA installed.
Then, you prepare runtime environment:

pip install -r requirements.txt

Use

Run

main.py(DA2Lite) runs with two main configurations like as follows:

CUDA_VISIBLE_DEVICES=0 python main.py --train_config_file=./configs/train/cifar10/cifar10/vgg16.yaml --compress_config_file=./configs/compress/tucker.yaml

The first one is train_config_file, which indicates training configurations and the other is compress_config_file, which represents compress configurations. The details of available configurations are described in Here.

After you run DA2Lite to compress a DNN model, logging and compressed model are saved in ./log directory.

The following shows the format of saving:

• YYYY-MM-DD.HH.MM.SS : format of saved directory for an instance.
  • models
    • origin_{dataset}_{model}.pt : The original model is saved.
    • compress_1_{dataset}_{model}.pt : The first compressed model is saved.
    • ...
  • process.log : The inevitable log is only logged.
  • specific_process.log : The training procedure log is added with process.log

Example

• Run the CIFAR10 example with resnet18 using tucker decomposition.
  • The pretrained-model are decomposed and right after fine-tuned: Here

Result

Cifar10 dataset

TODO

• Multi-GPU training
• PyTorchMobile conversion
• Train a model based on a custom dataset
• Rand-augmentation for improving an accuracy
• Make a model zoo
• Up-to-date model architectures.
• Train a model for object detection tasks (further future...)
• Compression methods for object detection tasks (further future...)

License

The entire codebase is under MIT license