Metabolic cost of information processing in Poisson variational autoencoders

TL;DR

This paper introduces a Poisson-based variational autoencoder model that links information processing with metabolic energy costs, highlighting a natural trade-off between coding accuracy and energy expenditure in neural systems.

Contribution

It presents a novel Poisson variational autoencoder framework that incorporates biologically inspired energy constraints, unlike traditional Gaussian VAEs, and demonstrates its effects on sparsity and activity.

Findings

Increasing the KL coefficient increases sparsity and reduces activity in P-VAE.

The metabolic cost structure is specific to Poisson statistics, not Gaussian VAEs.

Poisson variational inference offers a foundation for energy-aware computation theories.

Abstract

Computation in biological systems is fundamentally energy-constrained, yet standard theories of computation treat energy as freely available. Here, we argue that variational free energy minimization under a Poisson assumption offers a principled path toward an energy-aware theory of computation. Our key observation is that the Kullback-Leibler (KL) divergence term in the Poisson free energy objective becomes proportional to the prior firing rates of model neurons, yielding an emergent metabolic cost term that penalizes high baseline activity. This structure couples an abstract information-theoretic quantity -- the *coding rate* -- to a concrete biophysical variable -- the *firing rate* -- which enables a trade-off between coding fidelity and energy expenditure. Such a coupling arises naturally in the Poisson variational autoencoder (P-VAE) -- a brain-inspired generative model that encodes inputs as discrete spike counts and recovers a spiking form of *sparse coding* as a special case -- but is absent from standard Gaussian VAEs. To demonstrate that this metabolic cost structure is unique to the Poisson formulation, we compare the P-VAE against Grelu-VAE, a Gaussian VAE with ReLU rectification applied to latent samples, which controls for the non-negativity constraint. Across a systematic sweep of the KL term weighting coefficient $\beta$ and latent dimensionality, we find that increasing $\beta$ monotonically increases sparsity and reduces average spiking activity in the P-VAE. In contrast, Grelu-VAE representations remain unchanged, confirming that the effect is specific to Poisson statistics rather than a byproduct of non-negative representations. These results establish Poisson variational inference as a promising foundation for a resource-constrained theory of computation.