Pruning a neural network using Bayesian inference

TL;DR

This paper introduces a Bayesian inference-based method for neural network pruning that uses posterior probabilities and Bayes factors to guide iterative pruning, achieving sparsity with maintained accuracy.

Contribution

It presents a novel Bayesian approach to neural network pruning that integrates seamlessly with training and guides pruning decisions using Bayes factors.

Findings

Achieves high sparsity while maintaining accuracy

Effective across multiple benchmark datasets

Guides pruning with Bayesian posterior probabilities

Abstract

Neural network pruning is a highly effective technique aimed at reducing the computational and memory demands of large neural networks. In this research paper, we present a novel approach to pruning neural networks utilizing Bayesian inference, which can seamlessly integrate into the training procedure. Our proposed method leverages the posterior probabilities of the neural network prior to and following pruning, enabling the calculation of Bayes factors. The calculated Bayes factors guide the iterative pruning. Through comprehensive evaluations conducted on multiple benchmarks, we demonstrate that our method achieves desired levels of sparsity while maintaining competitive accuracy.