Dictionary Learning Using Rank-One Projection (ROP)

TL;DR

This paper introduces a novel Rank-One Projection (ROP) approach for dictionary learning that simplifies the optimization process to a single variable, guarantees convergence, and outperforms existing methods especially with limited data.

Contribution

The main contribution is formulating dictionary learning as an optimization over rank-one matrices, enabling a single-stage algorithm with guaranteed convergence.

Findings

ROP outperforms benchmark algorithms on synthetic data.

ROP performs well with small sample sizes.

The method reduces the number of tuning parameters.

Abstract

Dictionary learning aims to find a dictionary that can sparsely represent the training data. Methods in the literature typically formulate the dictionary learning problem as an optimisation with respect to two variables, i.e., dictionary and sparse coefficients, and solve it by alternating between two stages: sparse coding and dictionary update. The key contribution of this work is a Rank-One Projection (ROP) formulation where dictionary learning is cast as an optimisation with respect to a single variable which is a set of rank one matrices. The resulting algorithm is hence single staged. An alternating direction method of multipliers (ADMM) is derived to solve the optimisation problem and guarantees a global convergence despite non-convexity of the optimisation formulation. Also ROP reduces the number of tuning parameters required in other benchmark algorithms. Numerical tests demonstrate that ROP outperforms other benchmarks for both synthetic and real data especially when the sample number is small.