Exploring Action-Centric Representations Through the Lens of Rate-Distortion Theory

TL;DR

This paper investigates how action-centric representations can be understood through rate-distortion theory, showing they are efficient, task-specific compressions that do not aim for full data reconstruction, aligning with a goal-oriented perception approach.

Contribution

It provides a mathematical framework for action-centric representations using rate-distortion theory and demonstrates their efficiency and task-specific invariances via VAEs.

Findings

Representations are efficient lossy compressions.

They capture task-dependent invariances.

Full data reconstruction is unnecessary for optimal behavior.

Abstract

Organisms have to keep track of the information in the environment that is relevant for adaptive behaviour. Transmitting information in an economical and efficient way becomes crucial for limited-resourced agents living in high-dimensional environments. The efficient coding hypothesis claims that organisms seek to maximize the information about the sensory input in an efficient manner. Under Bayesian inference, this means that the role of the brain is to efficiently allocate resources in order to make predictions about the hidden states that cause sensory data. However, neither of those frameworks accounts for how that information is exploited downstream, leaving aside the action-oriented role of the perceptual system. Rate-distortion theory, which defines optimal lossy compression under constraints, has gained attention as a formal framework to explore goal-oriented efficient coding. In this work, we explore action-centric representations in the context of rate-distortion theory. We also provide a mathematical definition of abstractions and we argue that, as a summary of the relevant details, they can be used to fix the content of action-centric representations. We model action-centric representations using VAEs and we find that such representations i) are efficient lossy compressions of the data; ii) capture the task-dependent invariances necessary to achieve successful behaviour; and iii) are not in service of reconstructing the data. Thus, we conclude that full reconstruction of the data is rarely needed to achieve optimal behaviour, consistent with a teleological approach to perception.