Generic construction of scale-invariantly coarse grained memory

TL;DR

This paper proposes a universal method for constructing scale-invariant memory systems that encode past information efficiently, using Laplace transforms, to maximize predictive power for natural signals with scale-free fluctuations.

Contribution

It introduces a generic construction for scale-invariant coarse-grained memory based on Laplace transforms, revealing fundamental constraints for memory networks.

Findings

Memory encoding is equivalent to Laplace transform and its inverse.

The approach maximizes predictive information for scale-free signals.

Provides a fundamental framework for biological and artificial memory systems.

Abstract

Encoding temporal information from the recent past as spatially distributed activations is essential in order for the entire recent past to be simultaneously accessible. Any biological or synthetic agent that relies on the past to predict/plan the future, would be endowed with such a spatially distributed temporal memory. Simplistically, we would expect that resource limitations would demand the memory system to store only the most useful information for future prediction. For natural signals in real world which show scale free temporal fluctuations, the predictive information encoded in memory is maximal if the past information is scale invariantly coarse grained. Here we examine the general mechanism to construct a scale invariantly coarse grained memory system. Remarkably, the generic construction is equivalent to encoding the linear combinations of Laplace transform of the past information and their approximated inverses. This reveals a fundamental construction constraint on memory networks that attempt to maximize predictive information storage relevant to the natural world.