Convex Sparse Matrix Factorizations

TL;DR

This paper introduces a convex formulation for dictionary learning in sparse signal decomposition, replacing traditional bounds with a rank-reducing term, and compares its performance to non-convex methods using synthetic data.

Contribution

It proposes a novel convex approach to sparse matrix factorization that explicitly balances size and sparsity, with analysis of its estimation capabilities.

Findings

Convex formulation has a single local minimum.

Performance can be inferior to non-convex methods in some cases.

Comparison based on synthetic examples.

Abstract

We present a convex formulation of dictionary learning for sparse signal decomposition. Convexity is obtained by replacing the usual explicit upper bound on the dictionary size by a convex rank-reducing term similar to the trace norm. In particular, our formulation introduces an explicit trade-off between size and sparsity of the decomposition of rectangular matrices. Using a large set of synthetic examples, we compare the estimation abilities of the convex and non-convex approaches, showing that while the convex formulation has a single local minimum, this may lead in some cases to performance which is inferior to the local minima of the non-convex formulation.