On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling

TL;DR

This paper introduces a Fowler-Nordheim quantum-tunneling device that implements synaptic memory consolidation in neuromorphic AI, offering an ultra-energy-efficient solution that outperforms existing models in continual learning tasks.

Contribution

The paper presents the FN-synapse, a novel device that stores synaptic weights and usage history, enabling efficient, on-device memory consolidation for neuromorphic systems.

Findings

FN-synapse stores synaptic weight and usage history.

FN-synapse outperforms EWC in continual learning benchmarks.

Operates with femtojoules per update, indicating ultra-energy efficiency.

Abstract

Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunneling device can implement synaptic memory consolidation similar to what can be achieved by algorithmic consolidation models like the cascade and the elastic weight consolidation (EWC) models. The proposed FN-synapse not only stores the synaptic weight but also stores the synapse's historical usage statistic on the device itself. We also show that the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and we demonstrate that a network comprising FN-synapses outperforms a comparable EWC network for a small benchmark continual learning task. With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and persistent learning.