Efficient Stein Variational Inference for Reliable Distribution-lossless Network Pruning

TL;DR

This paper introduces DLLP, a novel Bayesian distribution-lossless network pruning method using Stein Variational Inference, which produces sparser, more reliable models without distribution truncation or human-crafted priors.

Contribution

The paper proposes DLLP, a new Bayesian pruning approach that models sub-networks as discrete priors and employs Stein Variational Inference to improve accuracy and reliability.

Findings

Achieves sparser networks with comparable or better accuracy.

Provides quantified reliability for pruned models.

Demonstrates effectiveness on CIFAR-10 and ImageNet datasets.

Abstract

Network pruning is a promising way to generate light but accurate models and enable their deployment on resource-limited edge devices. However, the current state-of-the-art assumes that the effective sub-network and the other superfluous parameters in the given network share the same distribution, where pruning inevitably involves a distribution truncation operation. They usually eliminate values near zero. While simple, it may not be the most appropriate method, as effective models may naturally have many small values associated with them. Removing near-zero values already embedded in model space may significantly reduce model accuracy. Another line of work has proposed to assign discrete prior over all possible sub-structures that still rely on human-crafted prior hypotheses. Worse still, existing methods use regularized point estimates, namely Hard Pruning, that can not provide error estimations and fail reliability justification for the pruned networks. In this paper, we propose a novel distribution-lossless pruning method, named DLLP, to theoretically find the pruned lottery within Bayesian treatment. Specifically, DLLP remodels the vanilla networks as discrete priors for the latent pruned model and the other redundancy. More importantly, DLLP uses Stein Variational Inference to approach the latent prior and effectively bypasses calculating KL divergence with unknown distribution. Extensive experiments based on small Cifar-10 and large-scaled ImageNet demonstrate that our method can obtain sparser networks with great generalization performance while providing quantified reliability for the pruned model.