Pruning for Feature-Preserving Circuits in CNNs

TL;DR

This paper introduces a novel method for extracting and visualizing feature-preserving circuits within CNNs, enabling better interpretability by isolating relevant convolutional kernels and creating modular, sparse representations of features.

Contribution

It presents a saliency-based pruning technique to extract modular circuits from CNNs that preserve specific features, along with visualization tools for interpretability.

Findings

Effective extraction of feature-preserving circuits using saliency criteria

Ability to create sparser sub-feature circuits that maintain feature responses

Development of visual circuit diagrams for better understanding of CNN filtering processes

Abstract

Deep convolutional neural networks are a powerful model class for a range of computer vision problems, but it is difficult to interpret the image filtering process they implement, given their sheer size. In this work, we introduce a method for extracting 'feature-preserving circuits' from deep CNNs, leveraging methods from saliency-based neural network pruning. These circuits are modular sub-functions, embedded within the network, containing only a subset of convolutional kernels relevant to a target feature. We compare the efficacy of 3 saliency-criteria for extracting these sparse circuits. Further, we show how 'sub-feature' circuits can be extracted, that preserve a feature's responses to particular images, dividing the feature into even sparser filtering processes. We also develop a tool for visualizing 'circuit diagrams', which render the entire image filtering process implemented by circuits in a parsable format.