Theory and learning protocols for the material tempotron model

TL;DR

This paper analyzes the tempotron neural model's capacity to decode temporal spike patterns, deriving theoretical limits and proposing efficient learning algorithms relevant for neuromorphic systems and biological understanding.

Contribution

It provides the first capacity calculation for the tempotron with discrete synapses and introduces two efficient learning algorithms approaching theoretical limits.

Findings

Capacity saturates information-theoretic bounds.

Learning algorithms achieve near-optimal association counts.

Protocols are suitable for neuromorphic hardware and biological models.

Abstract

Neural networks are able to extract information from the timing of spikes. Here we provide new results on the behavior of the simplest neuronal model which is able to decode information embedded in temporal spike patterns, the so called tempotron. Using statistical physics techniques we compute the capacity for the case of sparse, time-discretized input, and "material" discrete synapses, showing that the device saturates the information theoretic bounds with a statistics of output spikes that is consistent with the statistics of the inputs. We also derive two simple and highly efficient learning algorithms which are able to learn a number of associations which are close to the theoretical limit. The simplest versions of these algorithms correspond to distributed on-line protocols of interest for neuromorphic devices, and can be adapted to address the more biologically relevant continuous-time version of the classification problem, hopefully allowing for the understanding of some aspects of synaptic plasticity.