RGP: Neural Network Pruning through Its Regular Graph Structure

TL;DR

This paper introduces RGP, a novel one-shot neural network pruning method based on regular graph structures, achieving over 90% reduction in parameters and FLOPs while maintaining high accuracy.

Contribution

The paper proposes a new graph-based pruning approach that maps regular graphs to neural networks for efficient, one-shot pruning, differing from traditional importance-based methods.

Findings

RGP achieves over 90% parameter and FLOPs reduction.

Shorter average shortest path length correlates with higher accuracy.

RGP maintains high accuracy with minimal fine-tuning.

Abstract

Lightweight model design has become an important direction in the application of deep learning technology, pruning is an effective mean to achieve a large reduction in model parameters and FLOPs. The existing neural network pruning methods mostly start from the importance of parameters, and design parameter evaluation metrics to perform parameter pruning iteratively. These methods are not studied from the perspective of model topology, may be effective but not efficient, and requires completely different pruning for different datasets. In this paper, we study the graph structure of the neural network, and propose regular graph based pruning (RGP) to perform a one-shot neural network pruning. We generate a regular graph, set the node degree value of the graph to meet the pruning ratio, and reduce the average shortest path length of the graph by swapping the edges to obtain the optimal edge distribution. Finally, the obtained graph is mapped into a neural network structure to realize pruning. Experiments show that the average shortest path length of the graph is negatively correlated with the classification accuracy of the corresponding neural network, and the proposed RGP shows a strong precision retention capability with extremely high parameter reduction (more than 90%) and FLOPs reduction (more than 90%).