Multiscale nonlinear integration drives accurate encoding of input information

TL;DR

This study investigates how multiscale nonlinear integration enhances information encoding in biological and artificial systems, revealing fundamental processing features and optimal strategies across different timescales and system dimensions.

Contribution

It introduces a general model analyzing nonlinear summation and integration across multiple timescales, highlighting their roles in improving information transmission and discrimination.

Findings

Integration and fast processing increase mutual information.

High-dimensional embeddings and low-dimensional projections are optimal strategies.

Nonlinear operations enable tunable input discrimination.

Abstract

Biological and artificial systems encode information through several complex nonlinear operations, making their exact study a formidable challenge. These internal mechanisms often take place across multiple timescales and process external signals to enable functional output responses. In this work, we focus on two widely implemented paradigms: nonlinear summation, where signals are first processed independently and then combined; and nonlinear integration, where they are combined first and then processed. We study a general model where the input signal is propagated to an output unit through a processing layer via nonlinear activation functions. Further, we distinguish between the two cases of fast and slow processing timescales. We demonstrate that integration and fast-processing capabilities systematically enhance input-output mutual information over a wide range of parameters and system sizes, while simultaneously enabling tunable input discrimination. Moreover, we reveal that high-dimensional embeddings and low-dimensional projections emerge naturally as optimal competing strategies. Our results uncover the foundational features of nonlinear information processing with profound implications for both biological and artificial systems.