Analysis of Fast Structured Dictionary Learning

TL;DR

This paper analyzes the convergence properties of an alternating minimization algorithm for learning structured unitary sparsifying operators, demonstrating local linear convergence under mild assumptions and robustness in practice.

Contribution

It provides the first convergence analysis for this specific structured dictionary learning problem, including conditions for local linear convergence.

Findings

Algorithm converges to critical points generally.

Under mild assumptions, local linear convergence is established.

Algorithm is robust to initialization in practice.

Abstract

Sparsity-based models and techniques have been exploited in many signal processing and imaging applications. Data-driven methods based on dictionary and sparsifying transform learning enable learning rich image features from data, and can outperform analytical models. In particular, alternating optimization algorithms have been popular for learning such models. In this work, we focus on alternating minimization for a specific structured unitary sparsifying operator learning problem, and provide a convergence analysis. While the algorithm converges to the critical points of the problem generally, our analysis establishes under mild assumptions, the local linear convergence of the algorithm to the underlying sparsifying model of the data. Analysis and numerical simulations show that our assumptions hold for standard probabilistic data models. In practice, the algorithm is robust to initialization.