Predictive Sparse Manifold Transform

TL;DR

The paper introduces PSMT, a biologically plausible framework that learns and predicts natural dynamics using sparse coding and manifold learning, demonstrating improved future frame prediction in videos.

Contribution

It presents a novel two-layer framework combining sparse coding and manifold learning for dynamic prediction, with a focus on interpretability and biological plausibility.

Findings

PSMT outperforms static embedding baselines in future frame prediction.

Dynamic embedding space improves prediction accuracy.

The framework effectively captures topological and temporal structures in data.

Abstract

We present Predictive Sparse Manifold Transform (PSMT), a minimalistic, interpretable and biologically plausible framework for learning and predicting natural dynamics. PSMT incorporates two layers where the first sparse coding layer represents the input sequence as sparse coefficients over an overcomplete dictionary and the second manifold learning layer learns a geometric embedding space that captures topological similarity and dynamic temporal linearity in sparse coefficients. We apply PSMT on a natural video dataset and evaluate the reconstruction performance with respect to contextual variability, the number of sparse coding basis functions and training samples. We then interpret the dynamic topological organization in the embedding space. We next utilize PSMT to predict future frames compared with two baseline methods with a static embedding space. We demonstrate that PSMT with a dynamic embedding space can achieve better prediction performance compared to static baselines. Our work establishes that PSMT is an efficient unsupervised generative framework for prediction of future visual stimuli.