Online Multilinear Dictionary Learning

TL;DR

This paper introduces an online tensor dictionary learning method that leverages separable dictionaries and stochastic gradient techniques to achieve real-time performance and robustness, with demonstrated effectiveness on synthetic signals.

Contribution

It proposes a novel online tensor dictionary learning algorithm using separable dictionaries and stochastic gradient methods for improved stability and efficiency.

Findings

Achieves real-time tensor dictionary learning with reduced computational complexity.

Demonstrates robustness against bad initialization and outliers.

Shows impressive performance on synthetic signal experiments.

Abstract

A method for online tensor dictionary learning is proposed. With the assumption of separable dictionaries, tensor contraction is used to diminish a $N$-way model of $\mathcal{O}\left(L^N\right)$ into a simple matrix equation of $\mathcal{O}\left(NL^2\right)$ with a real-time capability. To avoid numerical instability due to inversion of sparse matrix, a class of stochastic gradient with memory is formulated via a least-square solution to guarantee convergence and robustness. Both gradient descent with exact line search and Newton's method are discussed and realized. Extensions onto how to deal with bad initialization and outliers are also explained in detail. Experiments on two synthetic signals confirms an impressive performance of our proposed method.