A Neural Network with Local Learning Rules for Minor Subspace Analysis

TL;DR

This paper introduces a biologically plausible neural network algorithm for minor subspace analysis (MSA) using local learning rules, filling a gap in neural signal processing tasks.

Contribution

It proposes a novel similarity matching objective and an adaptive algorithm for MSA, enabling local learning in neural networks for this task.

Findings

The proposed MSSM algorithm converges at a competitive rate.

The neural network implementation uses local learning rules.

The method effectively extracts the minor subspace in signal processing.

Abstract

The development of neuromorphic hardware and modeling of biological neural networks requires algorithms with local learning rules. Artificial neural networks using local learning rules to perform principal subspace analysis (PSA) and clustering have recently been derived from principled objective functions. However, no biologically plausible networks exist for minor subspace analysis (MSA), a fundamental signal processing task. MSA extracts the lowest-variance subspace of the input signal covariance matrix. Here, we introduce a novel similarity matching objective for extracting the minor subspace, Minor Subspace Similarity Matching (MSSM). Moreover, we derive an adaptive MSSM algorithm that naturally maps onto a novel neural network with local learning rules and gives numerical results showing that our method converges at a competitive rate.