Susceptibilities and Patterning: A Primer on Linear Response in Bayesian Learning

TL;DR

This paper introduces the theory of susceptibilities in Bayesian learning, linking data perturbations to model responses and providing tools for understanding neural network behavior and patterning.

Contribution

It develops a comprehensive framework for susceptibilities in Bayesian models, connecting them to the geometry of the loss landscape and patterning problems.

Findings

Susceptibility matrices relate data perturbations to model responses.

Empirical estimators for susceptibilities are detailed.

The theory connects statistical mechanics to neural network interpretability.

Abstract

These notes introduce the theory of susceptibilities as developed in [arXiv:2504.18274, arXiv:2601.12703] for interpreting neural networks. The susceptibility of an observable $\phi$ to a data perturbation is defined as a derivative of a posterior expectation, which by the fluctuation--dissipation theorem equals a posterior covariance. Different choices of $\phi$ yield different objects: per-sample losses give the influence matrix (the Bayesian influence function of [arXiv:2509.26544]), while component-localized observables give the structural susceptibility matrix that pairs model components with data patterns. The susceptibility matrix is (up to a factor of $n\beta$) the Jacobian of the map from data distributions to structural coordinates; its pseudo-inverse provides a linearized solution to the patterning problem of [arXiv:2601.13548]: finding data perturbations that produce a desired structural change. We motivate the theory from its statistical-mechanical foundations, then give a detailed exposition of susceptibilities, their empirical estimators, and their connection to the geometry of the loss landscape.