Normative Networks for Source Separation via Local Plasticity and Dendritic Computation

TL;DR

This paper introduces Predictive Entropy Maximization, a biologically plausible neural network approach for blind source separation that uses local learning rules and dendritic mechanisms to effectively recover structured and correlated sources.

Contribution

It proposes a novel online algorithm for source separation based on local plasticity and entropy approximation, improving biological plausibility and robustness over existing methods.

Findings

Outperforms existing biologically plausible algorithms in correlated source scenarios.

Remains robust under increased noise and source correlation.

Achieves competitive results with nonlocal, complex baselines.

Abstract

Blind source separation (BSS) is a natural framework for studying how latent causes may be recovered from sensory mixtures, but deriving online and biologically plausible algorithms for structured (i.e., constrained to known domains) and potentially correlated sources remains challenging. Recent work has derived neural networks for BSS from maximization of an entropy measure, yet its online implementations involve complex and nonlocal recurrent dynamics. Motivated by this perspective, we propose Predictive Entropy Maximization, which achieves competitive performance in BSS, using only local weight updates. The method employs a close approximation of an entropy measure, yielding an objective function with easily interpretable components. Minimizing this objective leads to a predictive neural architecture in which feedforward synapses follow an error-driven rule (that can be realized through dendritic mechanisms), lateral inhibitory connections are learned with local Hebbian plasticity, and source-domain constraints are enforced through simple output nonlinearities. We derive explicit spectral bounds on the surrogate error, characterizing when the approximation is accurate. Empirically, Predictive Entropy Maximization remains robust under increasing source correlation and observation noise, outperforms biologically plausible algorithms that rely on stronger independence or decorrelation assumptions, and remains competitive with exact determinant- and correlative-information-based baselines. These results show how local plasticity and adaptive lateral inhibition can emerge from maximizing a regularized second-order entropy over structured source domains. Our implementation code is available at https://github.com/BariscanBozkurt/Predictive-Entropy-Maximization.