A Clustering Approach to Learn Sparsely-Used Overcomplete Dictionaries

TL;DR

This paper presents an efficient clustering-based algorithm for learning overcomplete dictionaries in sparse coding, enabling approximate recovery of dictionary elements and improving accuracy through a subsequent refinement step.

Contribution

Introduces a novel clustering approach for dictionary learning that efficiently achieves approximate recovery in sparse coding scenarios.

Findings

Algorithm successfully recovers dictionary elements with high accuracy

Clustering method outperforms existing techniques in certain settings

Refinement step with $\,\ell_1$-regression enhances solution quality

Abstract

We consider the problem of learning overcomplete dictionaries in the context of sparse coding, where each sample selects a sparse subset of dictionary elements. Our main result is a strategy to approximately recover the unknown dictionary using an efficient algorithm. Our algorithm is a clustering-style procedure, where each cluster is used to estimate a dictionary element. The resulting solution can often be further cleaned up to obtain a high accuracy estimate, and we provide one simple scenario where $\ell_1$-regularized regression can be used for such a second stage.