AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates

TL;DR

AutoCompress is an automated structured pruning framework for DNNs that achieves ultra-high compression rates and significant inference speedups by combining advanced pruning schemes, an innovative purification step, and a guided heuristic search.

Contribution

It introduces an automatic structured pruning framework that effectively combines pruning schemes, employs an ADMM-based core algorithm with a novel purification step, and replaces deep reinforcement learning with a guided heuristic search.

Findings

Achieves up to 33x pruning rate and 120x parameter reduction at same accuracy.

Demonstrates significant inference speedup on smartphones.

Outperforms prior automatic model compression methods.

Abstract

Structured weight pruning is a representative model compression technique of DNNs to reduce the storage and computation requirements and accelerate inference. An automatic hyperparameter determination process is necessary due to the large number of flexible hyperparameters. This work proposes AutoCompress, an automatic structured pruning framework with the following key performance improvements: (i) effectively incorporate the combination of structured pruning schemes in the automatic process; (ii) adopt the state-of-art ADMM-based structured weight pruning as the core algorithm, and propose an innovative additional purification step for further weight reduction without accuracy loss; and (iii) develop effective heuristic search method enhanced by experience-based guided search, replacing the prior deep reinforcement learning technique which has underlying incompatibility with the target pruning problem. Extensive experiments on CIFAR-10 and ImageNet datasets demonstrate that AutoCompress is the key to achieve ultra-high pruning rates on the number of weights and FLOPs that cannot be achieved before. As an example, AutoCompress outperforms the prior work on automatic model compression by up to 33x in pruning rate (120x reduction in the actual parameter count) under the same accuracy. Significant inference speedup has been observed from the AutoCompress framework on actual measurements on smartphone. We release all models of this work at anonymous link: http://bit.ly/2VZ63dS.