When sufficiency is insufficient: the functional information bottleneck for identifying probabilistic neural representations

TL;DR

This paper introduces the functional information bottleneck framework to distinguish true probabilistic neural representations from heuristic codes, revealing that many neural networks previously thought to develop probabilistic codes do not actually do so.

Contribution

The paper proposes a novel information bottleneck method that evaluates neural representations based on sufficiency and minimality, clarifying criteria for probabilistic coding.

Findings

Many task-optimized neural networks do not develop minimal Bayesian posterior codes.

Neural networks often rely on heuristic input recoding rather than true probabilistic representations.

The framework provides a rigorous way to identify probabilistic neural codes in complex tasks.

Abstract

The neural basis of probabilistic computations remains elusive, even amidst growing evidence that humans and other animals track their uncertainty. Recent work has proposed that probabilistic representations arise naturally in task-optimized neural networks trained without explicitly probabilistic inductive biases. However, prior work has lacked clear criteria for distinguishing probabilistic representations, those that perform transformations characteristic of probabilistic computation, from heuristic neural codes that merely reformat inputs. We propose a novel information bottleneck framework, the functional information bottleneck (fIB), that crucially evaluates a neural representation based not only on its statistical sufficiency but also on its minimality, allowing us to disambiguate heuristic from probabilistic coding. To demonstrate the power of this framework, we study a variety of task-optimized neural networks that had been suggested to develop probabilistic representations in earlier work: networks trained to perform static inference tasks (such as cue combination and coordinate transformation) or dynamic state estimation tasks (Kalman filtering). In contrast to earlier claims, our minimality requirement reveals that probabilistic representations fail to emerge in these networks: they do not develop minimal codes of Bayesian posteriors in their hidden layer activities, and instead rely on heuristic input recoding. Therefore, it remains an open question under which conditions truly probabilistic representations emerge in neural networks. More generally, our work provides a stringent framework for identifying probabilistic neural codes. Thus, it lays the foundation for systematically examining whether, how, and which posteriors are represented in neural circuits during complex decision-making tasks.