Automatic Discovery of Visual Circuits

TL;DR

This paper presents a scalable method for automatically discovering interpretable subgraphs within deep vision models that correspond to specific visual concepts, enabling better understanding and robustness.

Contribution

It introduces a novel approach to identify and manipulate neural circuits related to visual concepts using minimal examples and functional connectivity analysis.

Findings

Extracted circuits causally influence model output

Editing circuits improves model robustness against adversarial attacks

Method scales to large pretrained models

Abstract

To date, most discoveries of network subcomponents that implement human-interpretable computations in deep vision models have involved close study of single units and large amounts of human labor. We explore scalable methods for extracting the subgraph of a vision model's computational graph that underlies recognition of a specific visual concept. We introduce a new method for identifying these subgraphs: specifying a visual concept using a few examples, and then tracing the interdependence of neuron activations across layers, or their functional connectivity. We find that our approach extracts circuits that causally affect model output, and that editing these circuits can defend large pretrained models from adversarial attacks.