Stable Lifelong Learning: Spiking neurons as a solution to instability in plastic neural networks

TL;DR

This paper investigates the instability issues in lifelong learning with plastic artificial neural networks and proposes spiking neurons as a solution to enhance stability over extended periods.

Contribution

It introduces the use of spiking neurons to address the long-term instability in plastic neural networks during lifelong learning.

Findings

Plasticity causes instability beyond training lifespan.

Instability leads to loss of reward-seeking behavior.

Spiking neurons mitigate instability in tested environments.

Abstract

Synaptic plasticity poses itself as a powerful method of self-regulated unsupervised learning in neural networks. A recent resurgence of interest has developed in utilizing Artificial Neural Networks (ANNs) together with synaptic plasticity for intra-lifetime learning. Plasticity has been shown to improve the learning capabilities of these networks in generalizing to novel environmental circumstances. However, the long-term stability of these trained networks has yet to be examined. This work demonstrates that utilizing plasticity together with ANNs leads to instability beyond the pre-specified lifespan used during training. This instability can lead to the dramatic decline of reward seeking behavior, or quickly lead to reaching environment terminal states. This behavior is shown to hold consistent for several plasticity rules on two different environments across many training time-horizons: a cart-pole balancing problem and a quadrupedal locomotion problem. We present a solution to this instability through the use of spiking neurons.