Relaxing in Warped Spaces: Generalized Hierarchical and Modular Dynamical Neural Network

TL;DR

This paper introduces a hierarchical, modular dynamical neural network model capable of learning and associating complex patterns by operating in warped spaces, with potential implications for understanding neural dynamics.

Contribution

It presents a novel neural network architecture derived from an energy function, capable of forming mappings and associations in warped, layered subspaces.

Findings

The model can form various two-dimensional mappings using periodic input signals.

It demonstrates the ability to relax state variables in designed warped spaces.

The framework suggests a certainty/uncertainty relation between input and output trajectories.

Abstract

We propose a dynamical neural network model with a hierarchical and modular structure. The network architecture can be derived by minimizing an energy function that is originally designed based on two kinds of neurons with quite different time constants. It has multiple subspaces that are spanned by neural parameters employed in the energy function, and adjacent subspaces are related to each other with a layered internetwork. Each internetwork further consists of a pair of a forward subnet and a backward one, and signals flowing through these subnets determine total dynamics of the network. The model can operate in either a learning or an association mode. In the learning mode, when periodic signals equivalent to repetitive neuronal bursting are suitably applied to input ports in all subspaces, mapping relationships corresponding to those input signals are eventually formed in internetworks between subspaces. Various two-dimensional mapping relationships between subspaces can be shaped by employing an appropriate set of periodic input signals with different frequencies based on the same mechanism as a Lissajous curve. The model in the association mode provides an overall framework such that state variables inside the network individually relax in warped spaces, each of which has been designed as favorable for a (or some) state variable(s). The association mode is further classified into two modes; unconstrained and constrained. In the latter mode, for instance, when a sufficiently slow periodic trajectory is set as an input, a warped output trajectory appears in each subspace as if imaginary layered networks with the inverse mapping relationships to existing forward subnets' were located hierarchically from outside to inside. These results suggest that a certainty/uncertainty relation exists between an input trajectory and an output trajectory.