Compressed Dictionary Learning
Karin Schnass, Flavio Teixeira

TL;DR
This paper introduces IcTKM, a fast dictionary learning algorithm that employs Johnson-Lindenstrauss-based dimensionality reduction with FFT, significantly reducing computational complexity while maintaining convergence and recovery guarantees.
Contribution
The paper proposes IcTKM, an efficient dictionary learning method that leverages fast Fourier transform-based Johnson-Lindenstrauss embeddings to reduce computational cost and ensure stable recovery.
Findings
IcTKM can recover incoherent overcomplete dictionaries with high probability.
Embedding dimension scales as O(S log^4 S log^3 K), enabling significant speedups.
Numerical simulations confirm IcTKM's effectiveness on high-dimensional data.
Abstract
In this paper we show that the computational complexity of the Iterative Thresholding and K-residual-Means (ITKrM) algorithm for dictionary learning can be significantly reduced by using dimensionality-reduction techniques based on the Johnson-Lindenstrauss lemma. The dimensionality reduction is efficiently carried out with the fast Fourier transform. We introduce the Iterative compressed-Thresholding and K-Means (IcTKM) algorithm for fast dictionary learning and study its convergence properties. We show that IcTKM can locally recover an incoherent, overcomplete generating dictionary of atoms from training signals of sparsity level with high probability. Fast dictionary learning is achieved by embedding the training data and the dictionary into dimensions, and recovery is shown to be locally stable with an embedding dimension which scales as low as $m = O(S \log^4 S…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
