Shapley Pruning for Neural Network Compression

TL;DR

This paper introduces a Shapley value-based framework for neural network pruning, providing practical approximations and a new benchmark, achieving state-of-the-art compression results.

Contribution

It develops a general Shapley value-based approach for CNN pruning, connecting existing concepts and introducing practical approximations and a new benchmark.

Findings

Achieves state-of-the-art network compression results.

Proposes practical Shapley value approximations.

Introduces a new Oracle rank benchmark.

Abstract

Neural network pruning is a rich field with a variety of approaches. In this work, we propose to connect the existing pruning concepts such as leave-one-out pruning and oracle pruning and develop them into a more general Shapley value-based framework that targets the compression of convolutional neural networks. To allow for practical applications in utilizing the Shapley value, this work presents the Shapley value approximations, and performs the comparative analysis in terms of cost-benefit utility for the neural network compression. The proposed ranks are evaluated against a new benchmark, Oracle rank, constructed based on oracle sets. The broad experiments show that the proposed normative ranking and its approximations show practical results, obtaining state-of-the-art network compression.