Boosting the Convergence of Reinforcement Learning-based Auto-pruning Using Historical Data

TL;DR

This paper introduces an efficient auto-pruning framework for neural networks that leverages historical data, transfer learning, and assistant learning to significantly accelerate reinforcement learning-based pruning processes.

Contribution

It proposes a novel auto-pruning framework that enhances RL convergence and training speed through transfer learning and auxiliary learning techniques.

Findings

Accelerates auto-pruning by 1.5-2.5 times for ResNet20

Achieves 1.81-2.375 times speedup on ResNet56, ResNet18, MobileNet v1

Improves sample efficiency and transferability of RL-based pruning

Abstract

Recently, neural network compression schemes like channel pruning have been widely used to reduce the model size and computational complexity of deep neural network (DNN) for applications in power-constrained scenarios such as embedded systems. Reinforcement learning (RL)-based auto-pruning has been further proposed to automate the DNN pruning process to avoid expensive hand-crafted work. However, the RL-based pruner involves a time-consuming training process and the high expense of each sample further exacerbates this problem. These impediments have greatly restricted the real-world application of RL-based auto-pruning. Thus, in this paper, we propose an efficient auto-pruning framework which solves this problem by taking advantage of the historical data from the previous auto-pruning process. In our framework, we first boost the convergence of the RL-pruner by transfer learning. Then, an augmented transfer learning scheme is proposed to further speed up the training process by improving the transferability. Finally, an assistant learning process is proposed to improve the sample efficiency of the RL agent. The experiments have shown that our framework can accelerate the auto-pruning process by 1.5-2.5 times for ResNet20, and 1.81-2.375 times for other neural networks like ResNet56, ResNet18, and MobileNet v1.