Storing cycles in Hopfield-type networks with pseudoinverse learning rule: admissibility and network topology

TL;DR

This paper investigates how cyclic patterns of neuronal activity can be stored in Hopfield-type networks using the pseudoinverse learning rule, focusing on the structural features and topology of admissible cycles.

Contribution

It characterizes admissible cycles via Fourier transform properties and classifies cycles based on their loop structure, linking them to specific network topologies.

Findings

Admissibility depends on the Fourier transform having exactly r nonzero columns.

Networks for simple cycles form feedforward chains with feedback to one neuron.

Simulations demonstrate successful cycle retrieval in Hopfield and spiking neuron networks.

Abstract

Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying the role of the network connectivities for generating cyclic patterns is fundamental for understanding the generation of rhythmic movements. In this paper, the storage of binary cycles in neural networks is investigated. We call a cycle $\Sigma$ admissible if a connectivity matrix satisfying the cycle's transition conditions exists, and construct it using the pseudoinverse learning rule. Our main focus is on the structural features of admissible cycles and corresponding network topology. We show that $\Sigma$ is admissible if and only if its discrete Fourier transform contains exactly $r={rank}(\Sigma)$ nonzero columns. Based on the decomposition of the rows of $\Sigma$ into loops, where a loop is the set of all cyclic permutations of a row, cycles are classified as simple cycles, separable or inseparable composite cycles. Simple cycles contain rows from one loop only, and the network topology is a feedforward chain with feedback to one neuron if the loop-vectors in $\Sigma$ are cyclic permutations of each other. Composite cycles contain rows from at least two disjoint loops, and the neurons corresponding to the rows in $\Sigma$ from the same loop are identified with a cluster. Networks constructed from separable composite cycles decompose into completely isolated clusters. For inseparable composite cycles at least two clusters are connected, and the cluster-connectivity is related to the intersections of the spaces spanned by the loop-vectors of the clusters. Simulations showing successfully retrieved cycles in continuous-time Hopfield-type networks and in networks of spiking neurons are presented.