Matrix Measure Flows: A Novel Approach to Stable Plasticity in Neural Networks

TL;DR

This paper introduces matrix measure flows as a new analytical tool to assess the stability of neural networks with dynamic, plastic synapses, enabling systematic stability analysis in biologically inspired and machine learning models.

Contribution

It proposes the concept of matrix measure flows for stability analysis of neural networks with time-varying weights, bridging neuroscience and machine learning.

Findings

Matrix measure flow obeys a simple inequality for certain matrix flows.

Allows stability and contraction analysis of recurrent neural networks with plastic synapses.

Supports robustness analysis in neural dynamics related to optimization and control.

Abstract

This letter introduces the notion of a matrix measure flow as a tool for analyzing the stability of neural networks with time-varying weights. Given a matrix flow -- for example, one induced by gradient-based adaptation -- the matrix measure flow tracks the progression of an associated matrix measure (or logarithmic norm). We show that for certain matrix flows of interest in computational neuroscience and machine learning, the associated matrix measure flow obeys a simple inequality. In the context of neuroscience, synapses -- the connections between neurons in the brain -- are constantly being updated. This plasticity subserves many important functions, such as memory consolidation and fast parameter adaptation towards real-time control objectives. However, synaptic plasticity poses a challenge for stability analyses of recurrent neural networks, which typically assume fixed synapses. Matrix measure flows allow the stability and contraction properties of recurrent neural networks with plastic synapses to be systematically analyzed. This in turn can be used to establish the robustness properties of neural dynamics, including those associated with problems in optimization and control. We consider examples of synapses subject to Hebbian and/or Anti-Hebbian plasticity, as well as covariance-based and gradient-based rules.