# Energy--Information Trade-off Induces Continuous and Discontinuous Phase Transitions in Lateral Predictive Coding

## TL;DR

This paper analyzes how energy and information trade-offs in lateral predictive coding lead to phase transitions in neural network structures, revealing insights into the emergence of complex internal models.

## Contribution

It provides an analytical and numerical study of phase transitions in lateral predictive coding, highlighting the effects of energy-information trade-offs on network structure and efficiency.

## Key findings

- Identification of continuous and discontinuous phase transitions in synaptic weights
- Discovery of a reciprocity-breaking transition building cyclic dominance
- Optimal networks saturate energy efficiency bounds

## Abstract

Lateral predictive coding is a recurrent neural network which creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs. Here we investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding analytically and by numerical minimization of a free energy. We observe several phase transitions in the synaptic weight matrix, especially a continuous transition which breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions. The optimal network follows an ideal-gas law in an extended temperature range and saturates the efficiency upper-bound of energy utilization. These results bring theoretical insights on the emergence and evolution of complex internal models in predictive processing systems.