# Predictive remapping and allocentric coding as consequences of energy efficiency in recurrent neural network models of active vision

## TL;DR

This paper shows that complex brain functions like stable vision during eye movements can emerge from simple energy-saving rules in neural networks.

## Contribution

The study demonstrates that predictive remapping and allocentric coding can arise from energy efficiency optimization in recurrent neural networks.

## Key findings

- Predictive remapping emerges from energy efficiency optimization in the model.
- An allocentric reference frame is learned to guide visual predictions.
- Energy efficiency alone can lead to complex neural computations.

## Abstract

Despite moving our eyes from one location to another, our perception of the world is stable—an aspect thought to rely on predictive computations that use efference copies to predict the upcoming foveal input. Are these complex computations and required connectivity scaffolds genetically encoded, or could they emerge from simpler principles? Here, we consider the organism’s limited energy budget as a potential origin. We expose a recurrent neural network to sequences of fixation patches and saccadic efference copies, training the model to minimize energy consumption (preactivation). We show that targeted inhibitory predictive remapping emerges from this energy-efficiency optimization alone. Furthermore, this computation relies on the model’s learned ability to re-code egocentric eye coordinates into an allocentric (image-centric) reference frame. Together, our findings suggest that both allocentric coding and predictive remapping can emerge from energy-efficiency constraints, demonstrating how complex neural computations can arise from simple physical principles.

•Neural networks are optimized for energy efficiency during active vision•Predictive remapping emerges as a consequence of energy efficiency•An allocentric reference frame emerges that guides predictions

Neural networks are optimized for energy efficiency during active vision

Predictive remapping emerges as a consequence of energy efficiency

An allocentric reference frame emerges that guides predictions

We show how some of the brain’s amazing abilities, such as keeping our view of the world stable even when our eyes move, might come from simple ideas such as saving energy. Instead of assuming that the brain is wired with complex instructions for predicting what we will see next, we explored whether these skills could develop naturally from basic physical principles. We apply a computer model that mimics how our eyes move and how the brain processes visual information. This model was trained to perform eye movements while trying to use as little energy as possible by reducing unnecessary neural activity. Surprisingly, as the model learned to be more energy efficient, it started to develop a process called predictive remapping. This process is how the brain predicts what will appear in our vision after an eye movement, so our perception stays smooth and stable. Moreover, the model learned to create an internal representation that translates the position of the eyes into a more stable, environment-centered frame of reference. This internal map helps the system predict future visual input and decide when to inhibit or reduce certain signals, making the whole process more efficient. Altogether, we show that complex visual functions such as predictive remapping and creating an environment-centered reference frame can emerge naturally when a system is optimized for energy efficiency.

This study gives an example of how complex computations in neural networks can emerge from simple physical principles. Training a model to optimize internal energy efficiency while performing eye movements suffices for predictive remapping to emerge. The model learns to translate eye movements into an allocentric reference frame. Based on this reference frame, it learns to predict and inhibit the next fixation.