Group Invariant Dictionary Learning

TL;DR

This paper introduces a framework for learning dictionaries that are invariant under symmetries in data, leveraging group actions and convex penalties, with applications to convolutional and shift-invariant dictionary learning.

Contribution

It develops a novel convex optimization approach for learning symmetry-invariant dictionaries, extending to continuous shifts using Toeplitz matrices.

Findings

Incorporating symmetries improves dictionary learning on small datasets.

The method effectively handles invariance under group actions, including continuous shifts.

Numerical experiments demonstrate advantages over traditional methods when symmetries are present.

Abstract

The dictionary learning problem concerns the task of representing data as sparse linear sums drawn from a smaller collection of basic building blocks. In application domains where such techniques are deployed, we frequently encounter datasets where some form of symmetry or invariance is present. Motivated by this observation, we develop a framework for learning dictionaries for data under the constraint that the collection of basic building blocks remains invariant under such symmetries. Our procedure for learning such dictionaries relies on representing the symmetry as the action of a matrix group acting on the data, and subsequently introducing a convex penalty function so as to induce sparsity with respect to the collection of matrix group elements. Our framework specializes to the convolutional dictionary learning problem when we consider integer shifts. Using properties of positive semidefinite Hermitian Toeplitz matrices, we develop an extension that learns dictionaries that are invariant under continuous shifts. Our numerical experiments on synthetic data and ECG data show that the incorporation of such symmetries as priors are most valuable when the dataset has few data-points, or when the full range of symmetries is inadequately expressed in the dataset.