Analysis of Fast Alternating Minimization for Structured Dictionary Learning

TL;DR

This paper analyzes the convergence properties of alternating minimization algorithms for structured (unitary) dictionary learning, demonstrating rapid local convergence and robustness to initialization in image processing applications.

Contribution

It provides the first theoretical proof of rapid local linear convergence for structured dictionary learning algorithms under mild conditions.

Findings

Algorithm converges to stationary points in general.

Proven rapid local linear convergence to the true model.

Algorithm is robust to different initializations.

Abstract

Methods exploiting sparsity have been popular in imaging and signal processing applications including compression, denoising, and imaging inverse problems. Data-driven approaches such as dictionary learning and transform learning enable one to discover complex image features from datasets and provide promising performance over analytical models. Alternating minimization algorithms have been particularly popular in dictionary or transform learning. In this work, we study the properties of alternating minimization for structured (unitary) sparsifying operator learning. While the algorithm converges to the stationary points of the non-convex problem in general, we prove rapid local linear convergence to the underlying generative model under mild assumptions. Our experiments show that the unitary operator learning algorithm is robust to initialization.