Unlocking the potential of two-point cells for energy-efficient and resilient training of deep nets

TL;DR

This paper introduces a novel deep neural network architecture driven by context-sensitive two-point layer 5 pyramidal cells, achieving significant energy efficiency and resilience in processing large heterogeneous audio-visual data, with promising implications for neuromorphic computing.

Contribution

The paper demonstrates for the first time how L5PCs-driven DNNs can process real-world multimodal data with substantially reduced energy consumption and increased robustness, introducing the MCC architecture.

Findings

Energy savings up to 62% in semi-supervised setup

Potential for 1250x less energy in supervised learning

Fast learning and resilience to neural damage

Abstract

Context-sensitive two-point layer 5 pyramidal cells (L5PCs) were discovered as long ago as 1999. However, the potential of this discovery to provide useful neural computation has yet to be demonstrated. Here we show for the first time how a transformative L5PCs-driven deep neural network (DNN), termed the multisensory cooperative computing (MCC) architecture, can effectively process large amounts of heterogeneous real-world audio-visual (AV) data, using far less energy compared to best available 'point' neuron-driven DNNs. A novel highly-distributed parallel implementation on a Xilinx UltraScale+ MPSoC device estimates energy savings up to 245759 $ \times $ 50000 $\mu$J (i.e., 62% less than the baseline model in a semi-supervised learning setup) where a single synapse consumes $8e^{-5}\mu$J. In a supervised learning setup, the energy-saving can potentially reach up to 1250x less (per feedforward transmission) than the baseline model. The significantly reduced neural activity in MCC leads to inherently fast learning and resilience against sudden neural damage. This remarkable performance in pilot experiments demonstrates the embodied neuromorphic intelligence of our proposed cooperative L5PC that receives input from diverse neighbouring neurons as context to amplify the transmission of most salient and relevant information for onward transmission, from overwhelmingly large multimodal information utilised at the early stages of on-chip training. Our proposed approach opens new cross-disciplinary avenues for future on-chip DNN training implementations and posits a radical shift in current neuromorphic computing paradigms.