Confident magnitude-based neural network pruning

TL;DR

This paper introduces a method for neural network pruning that incorporates rigorous uncertainty quantification, providing statistical guarantees for maintaining performance while reducing model size in computer vision tasks.

Contribution

It presents a novel approach combining neural network pruning with distribution-free uncertainty quantification to ensure reliable model compression.

Findings

Provides finite-sample statistical guarantees for pruning

Demonstrates effective uncertainty-aware pruning in computer vision

Maintains high performance with significantly reduced models

Abstract

Pruning neural networks has proven to be a successful approach to increase the efficiency and reduce the memory storage of deep learning models without compromising performance. Previous literature has shown that it is possible to achieve a sizable reduction in the number of parameters of a deep neural network without deteriorating its predictive capacity in one-shot pruning regimes. Our work builds beyond this background in order to provide rigorous uncertainty quantification for pruning neural networks reliably, which has not been addressed to a great extent in previous literature focusing on pruning methods in computer vision settings. We leverage recent techniques on distribution-free uncertainty quantification to provide finite-sample statistical guarantees to compress deep neural networks, while maintaining high performance. Moreover, this work presents experiments in computer vision tasks to illustrate how uncertainty-aware pruning is a useful approach to deploy sparse neural networks safely.