Bio-Inspired Mamba: Temporal Locality and Bioplausible Learning in Selective State Space Models

TL;DR

Bio-Inspired Mamba (BIM) is a novel online learning framework that combines biological learning principles with the Mamba architecture, enabling efficient training of spiking neural networks for complex temporal tasks.

Contribution

This paper introduces BIM, integrating RTRL and STDP-like rules for biologically plausible, efficient learning in selective state space models, addressing temporal locality and long-range dependencies.

Findings

Competitive performance on language and speech tasks

Improved energy efficiency and neuromorphic suitability

Enhanced understanding of biological temporal processing

Abstract

This paper introduces Bio-Inspired Mamba (BIM), a novel online learning framework for selective state space models that integrates biological learning principles with the Mamba architecture. BIM combines Real-Time Recurrent Learning (RTRL) with Spike-Timing-Dependent Plasticity (STDP)-like local learning rules, addressing the challenges of temporal locality and biological plausibility in training spiking neural networks. Our approach leverages the inherent connection between backpropagation through time and STDP, offering a computationally efficient alternative that maintains the ability to capture long-range dependencies. We evaluate BIM on language modeling, speech recognition, and biomedical signal analysis tasks, demonstrating competitive performance against traditional methods while adhering to biological learning principles. Results show improved energy efficiency and potential for neuromorphic hardware implementation. BIM not only advances the field of biologically plausible machine learning but also provides insights into the mechanisms of temporal information processing in biological neural networks.