Gradient-based Training of Slow Feature Analysis by Differentiable Approximate Whitening

TL;DR

Power Slow Feature Analysis (PowerSFA) introduces a gradient-based approach for extracting slow features from high-dimensional, fast-varying data, enabling end-to-end training of complex models for meaningful low-dimensional feature extraction.

Contribution

It presents PowerSFA, a novel gradient-based method that extends SFA to arbitrary differentiable architectures for effective slow feature extraction.

Findings

Successfully extracts meaningful features from synthetic data

Effective on ego-visual data

Works on datasets with symmetric non-temporal similarities

Abstract

We propose Power Slow Feature Analysis, a gradient-based method to extract temporally slow features from a high-dimensional input stream that varies on a faster time-scale, as a variant of Slow Feature Analysis (SFA) that allows end-to-end training of arbitrary differentiable architectures and thereby significantly extends the class of models that can effectively be used for slow feature extraction. We provide experimental evidence that PowerSFA is able to extract meaningful and informative low-dimensional features in the case of (a) synthetic low-dimensional data, (b) ego-visual data, and also for (c) a general dataset for which symmetric non-temporal similarities between points can be defined.