# Progressive Gradient Pruning for Classification, Detection and DomainAdaptation

## TL;DR

This paper introduces Progressive Gradient Pruning, a novel iterative filter pruning method during training that improves neural network efficiency while maintaining accuracy, applicable to classification, detection, and domain adaptation tasks.

## Contribution

The paper presents a new pruning technique with a novel filter selection criterion and adaptive momentum strategies, outperforming existing methods in efficiency and accuracy trade-offs.

## Key findings

- Achieves better accuracy-complexity trade-off than existing methods
- Effective for classification, detection, and domain adaptation
- Reduces network size and energy consumption during training

## Abstract

Although deep neural networks (NNs) have achievedstate-of-the-art accuracy in many visual recognition tasks,the growing computational complexity and energy con-sumption of networks remains an issue, especially for ap-plications on platforms with limited resources and requir-ing real-time processing. Filter pruning techniques haverecently shown promising results for the compression andacceleration of convolutional NNs (CNNs). However, thesetechniques involve numerous steps and complex optimisa-tions because some only prune after training CNNs, whileothers prune from scratch during training by integratingsparsity constraints or modifying the loss function.In this paper we propose a new Progressive GradientPruning (PGP) technique for iterative filter pruning dur-ing training. In contrast to previous progressive pruningtechniques, it relies on a novel filter selection criterion thatmeasures the change in filter weights, uses a new hard andsoft pruning strategy and effectively adapts momentum ten-sors during the backward propagation pass. Experimentalresults obtained after training various CNNs on image datafor classification, object detection and domain adaptationbenchmarks indicate that the PGP technique can achievea better trade-off between classification accuracy and net-work (time and memory) complexity than PSFP and otherstate-of-the-art filter pruning techniques.