D-Score: A Synapse-Inspired Approach for Filter Pruning

TL;DR

This paper proposes D-Score, a novel filter pruning method inspired by synaptic transmission, which evaluates positive and negative weights separately to identify and prune less important filters in CNNs, reducing computational costs.

Contribution

The paper introduces a synapse-inspired filter importance metric, D-Score, that improves pruning effectiveness by analyzing positive and negative weights independently.

Findings

Reduces FLOPs and Params significantly on CIFAR-10 and ImageNet.

Maintains accuracy with minimal performance drop.

Demonstrates effectiveness of neuroscience-inspired pruning approach.

Abstract

This paper introduces a new aspect for determining the rank of the unimportant filters for filter pruning on convolutional neural networks (CNNs). In the human synaptic system, there are two important channels known as excitatory and inhibitory neurotransmitters that transmit a signal from a neuron to a cell. Adopting the neuroscientific perspective, we propose a synapse-inspired filter pruning method, namely Dynamic Score (D-Score). D-Score analyzes the independent importance of positive and negative weights in the filters and ranks the independent importance by assigning scores. Filters having low overall scores, and thus low impact on the accuracy of neural networks are pruned. The experimental results on CIFAR-10 and ImageNet datasets demonstrate the effectiveness of our proposed method by reducing notable amounts of FLOPs and Params without significant Acc. Drop.