Learning temporal structure of the input with a network of integrate-and-fire neurons

TL;DR

This paper investigates how a network of integrate-and-fire neurons can learn and represent the temporal structure of input signals efficiently by adjusting synaptic weights, especially when the input follows specific temporal dynamics.

Contribution

It introduces a novel approach where the network learns to encode the temporal dynamics of input signals, extending previous work focused on low-dimensional input structure.

Findings

Network achieves efficient representation when input changes slowly.

Increasing network dimensionality improves representation for linear differential inputs.

Proposed learning rule enables synaptic weights to adapt for optimal input encoding.

Abstract

The task of the brain is to look for structure in the external input. We study a network of integrate-and-fire neurons with several types of recurrent connections that learns the structure of its time-varying feedforward input by attempting to efficiently represent this input with spikes. The efficiency of the representation arises from incorporating the structure of the input into the decoder, which is implicit in the learned synaptic connectivity of the network. While in the original work of [Boerlin, Machens, Den\`eve 2013] and [Brendel et al., 2017] the structure learned by the network to make the representation efficient was the low-dimensionality of the feedforward input, in the present work it is its temporal dynamics. The network achieves the efficiency by adjusting its synaptic weights in such a way, that for any neuron in the network, the recurrent input cancels the feedforward for most of the time. We show that if the only temporal structure that the input possesses is that it changes slowly on the time scale of neuronal integration, the dimensionality of the network dynamics is equal to the dimensionality of the input. However, if the input follows a linear differential equation of the first order, the efficiency of the representation can be increased by increasing the dimensionality of the network dynamics in comparison to the dimensionality of the input. If there is only one type of slow synaptic current in the network, the increase is two-fold, while if there are two types of slow synaptic currents that decay with different rates and whose amplitudes can be adjusted separately, it is advantageous to make the increase three-fold. We numerically simulate the network with synaptic weights that imply the most efficient input representation in the above cases. We also propose a learning rule by means of which the corresponding synaptic weights can be learned.