Efficient Sum of Outer Products Dictionary Learning (SOUP-DIL) - The $\ell_0$ Method

TL;DR

This paper introduces an efficient $oldsymbol{ ext{l}}_0$-based dictionary learning method called SOUP-DIL, which approximates data with sparse rank-one matrices and uses a block coordinate descent algorithm with closed-form solutions, achieving faster computation and promising results.

Contribution

It proposes a novel $oldsymbol{ ext{l}}_0$ dictionary learning algorithm using sum of outer products and block coordinate descent with closed-form solutions, improving efficiency over traditional methods.

Findings

Significant speed-ups over K-SVD in sparse representation and denoising.

Convergence analysis of the proposed algorithm.

Promising performance demonstrated through numerical experiments.

Abstract

The sparsity of natural signals and images in a transform domain or dictionary has been extensively exploited in several applications such as compression, denoising and inverse problems. More recently, data-driven adaptation of synthesis dictionaries has shown promise in many applications compared to fixed or analytical dictionary models. However, dictionary learning problems are typically non-convex and NP-hard, and the usual alternating minimization approaches for these problems are often computationally expensive, with the computations dominated by the NP-hard synthesis sparse coding step. In this work, we investigate an efficient method for $\ell_{0}$ "norm"-based dictionary learning by first approximating the training data set with a sum of sparse rank-one matrices and then using a block coordinate descent approach to estimate the unknowns. The proposed block coordinate descent algorithm involves efficient closed-form solutions. In particular, the sparse coding step involves a simple form of thresholding. We provide a convergence analysis for the proposed block coordinate descent approach. Our numerical experiments show the promising performance and significant speed-ups provided by our method over the classical K-SVD scheme in sparse signal representation and image denoising.