On the Invariance of Dictionary Learning and Sparse Representation to Projecting Data to a Discriminative Space

TL;DR

This paper proves that dictionary learning and sparse representation are invariant to linear transformations, including projections into discriminative spaces, supporting the approach of using discriminative bases directly as dictionaries.

Contribution

It establishes the invariance of dictionary learning to linear transformations, endorsing the use of discriminative bases as dictionaries without needing to learn in the projected space.

Findings

Dictionary learning is invariant to linear transformations.

Discriminative bases can be used directly as dictionaries.

Supports learning dictionaries in the original space for discriminative tasks.

Abstract

In this paper, it is proved that dictionary learning and sparse representation is invariant to a linear transformation. It subsumes the special case of transforming/projecting the data into a discriminative space. This is important because recently, supervised dictionary learning algorithms have been proposed, which suggest to include the category information into the learning of dictionary to improve its discriminative power. Among them, there are some approaches that propose to learn the dictionary in a discriminative projected space. To this end, two approaches have been proposed: first, assigning the discriminative basis as the dictionary and second, perform dictionary learning in the projected space. Based on the invariance of dictionary learning to any transformation in general, and to a discriminative space in particular, we advocate the first approach.