Scalable Convolutional Dictionary Learning with Constrained Recurrent Sparse Auto-encoders

TL;DR

This paper introduces CRsAE, a novel auto-encoder architecture that effectively recovers convolutional dictionaries from noisy signals, enabling blind source separation such as spike sorting in neuroscience.

Contribution

The paper presents the constrained recurrent sparse auto-encoder (CRsAE), integrating dictionary constraints into auto-encoder training for improved dictionary learning and source separation.

Findings

CRsAE successfully recovers underlying dictionaries from noisy signals.

The framework effectively separates sources in spike sorting applications.

Sensitivity analysis shows robustness to different SNR levels.

Abstract

Given a convolutional dictionary underlying a set of observed signals, can a carefully designed auto-encoder recover the dictionary in the presence of noise? We introduce an auto-encoder architecture, termed constrained recurrent sparse auto-encoder (CRsAE), that answers this question in the affirmative. Given an input signal and an approximate dictionary, the encoder finds a sparse approximation using FISTA. The decoder reconstructs the signal by applying the dictionary to the output of the encoder. The encoder and decoder in CRsAE parallel the sparse-coding and dictionary update steps in optimization-based alternating-minimization schemes for dictionary learning. As such, the parameters of the encoder and decoder are not independent, a constraint which we enforce for the first time. We derive the back-propagation algorithm for CRsAE. CRsAE is a framework for blind source separation that, only knowing the number of sources (dictionary elements), and assuming sparsely-many can overlap, is able to separate them. We demonstrate its utility in the context of spike sorting, a source separation problem in computational neuroscience. We demonstrate the ability of CRsAE to recover the underlying dictionary and characterize its sensitivity as a function of SNR.