Deep Convolutional Neural Networks Structured Pruning via Gravity Regularization

TL;DR

This paper introduces a physics-inspired gravity regularization method for structured pruning of deep convolutional neural networks, enabling efficient filter importance ranking and removal without complex modifications or extensive fine-tuning.

Contribution

The novel approach applies gravity concepts during training to automatically identify and prune less important filters in DCNNs, simplifying the pruning process.

Findings

Achieved competitive accuracy on CIFAR dataset

Reduced model complexity without extensive fine-tuning

Eliminated need for architecture modifications

Abstract

Structured pruning is a widely employed strategy for accelerating deep convolutional neural networks (DCNNs). However, existing methods often necessitate modifications to the original architectures, involve complex implementations, and require lengthy fine-tuning stages. To address these challenges, we propose a novel physics-inspired approach that integrates the concept of gravity into the training stage of DCNNs. In this approach, the gravity is directly proportional to the product of the masses of the convolution filter and the attracting filter, and inversely proportional to the square of the distance between them. We applied this force to the convolution filters, either drawing filters closer to the attracting filter (experiencing weaker gravity) toward non-zero weights or pulling filters farther away (subject to stronger gravity) toward zero weights. As a result, filters experiencing stronger gravity have their weights reduced to zero, enabling their removal, while filters under weaker gravity retain significant weights and preserve important information. Our method simultaneously optimizes the filter weights and ranks their importance, eliminating the need for complex implementations or extensive fine-tuning. We validated the proposed approach on popular DCNN architectures using the CIFAR dataset, achieving competitive results compared to existing methods.