Neural network model with discrete and continuous information representation

TL;DR

This paper introduces a neural network model capable of representing both discrete and continuous information by combining features of associative memory and Mexican-hat interaction models, analyzed through statistical mechanics.

Contribution

It proposes a novel neural network model that achieves both discrete and continuous information representation and identifies conditions for localized retrieval phases.

Findings

Localized retrieval phase exists in the proposed model

Orthogonality of patterns enables discrete representation

Neutral stability along positions supports continuous representation

Abstract

An associative memory model and a neural network model with a Mexican-hat type interaction are the two most typical attractor networks used in the artificial neural network models. The associative memory model has discretely distributed fixed-point attractors, and achieves a discrete information representation. On the other hand, a neural network model with a Mexican-hat type interaction uses a line attractor to achieves a continuous information representation, which can be seen in the working memory in the prefrontal cortex and columnar activity in the visual cortex. In the present study, we propose a neural network model that achieves discrete and continuous information representation. We use a statistical-mechanical analysis to find that a localized retrieval phase exists in the proposed model, where the memory pattern is retrieved in the localized subpopulation of the network. In the localized retrieval phase, the discrete and continuous information representation is achieved by using the orthogonality of the memory patterns and the neutral stability of fixed points along the positions of the localized retrieval. The obtained phase diagram suggests that the antiferromagnetic interaction and the external field are important for generating the localized retrieval phase.