Overcomplete Dictionary Learning with Jacobi Atom Updates

TL;DR

This paper introduces a parallel Jacobi-based approach to dictionary learning for sparse representations, demonstrating improved performance over traditional sequential methods in image processing tasks.

Contribution

It proposes a novel parallel Jacobi atom update algorithm for dictionary learning, enhancing efficiency and effectiveness over existing sequential methods.

Findings

Parallel algorithms outperform sequential ones in image representation tasks.

Simultaneous atom updates lead to better dictionaries.

Numerical experiments validate the superiority of the proposed method.

Abstract

Dictionary learning for sparse representations is traditionally approached with sequential atom updates, in which an optimized atom is used immediately for the optimization of the next atoms. We propose instead a Jacobi version, in which groups of atoms are updated independently, in parallel. Extensive numerical evidence for sparse image representation shows that the parallel algorithms, especially when all atoms are updated simultaneously, give better dictionaries than their sequential counterparts.