Low-resolution descriptions of model neural activity reveal hidden features and underlying system properties

TL;DR

This paper introduces a method to analyze neural network models using low-resolution representations, revealing hidden features and system properties by optimizing the selection of neuron subgroups based on information content.

Contribution

It develops the MEOW workflow for unsupervised identification of informative neuron subgroups in a Hopfield model, linking low-resolution analysis to underlying system properties.

Findings

Optimal neuron mappings depend on interaction matrix properties.

Low-resolution representations can reveal hidden features.

The method enables quantitative insights into complex systems.

Abstract

The analysis of complex systems such as neural networks is made particularly difficult by the overwhelming number of their interacting components. In the absence of prior knowledge, identifying a small but informative subset of network nodes on which the analysis should focus is a rather challenging task. In this work, we address this problem in the context of a Hopfield model, which is observed through the lenses of low-resolution representations, or decimation mappings, consisting of subgroups of its neurons. The optimal, most informative mappings of the network are defined through a recently developed methodology, the mapping entropy optimisation workflow (MEOW), which performs an unsupervised analysis of the states sampled by the network and identifies those subgroups of spins whose configuration distribution is closest to that of the full, high-resolution model. Which neurons are retained in an optimal mapping is found to critically depend on the properties of the interaction matrix of the network and the level of detail employed to describe the system; by these means, it is thus possible to extract quantitative insight about the underlying properties of the high-resolution model through the analysis of its optimal low-resolution representations. These results show a tight and potentially fruitful relation between the level of detail at which the network is inspected and the type and amount of information that can be gathered from it, and showcase the MEOW approach as a practical, enabling tool for the study of complex systems.