Convergence Rate Analysis of the Majorize-Minimize Subspace Algorithm -- Extended Version

TL;DR

This paper derives theoretical convergence rates for the Majorize-Minimize (MM) subspace algorithm, a promising optimization method in signal and image processing, analyzing both batch and online versions and the impact of subspace selection.

Contribution

It provides the first theoretical convergence rate analysis for the MM subspace algorithm, including effects of subspace choice and different implementation modes.

Findings

Convergence rates are established for both batch and online MM subspace algorithms.

The influence of subspace selection on convergence speed is characterized.

The results enhance understanding of MM algorithm performance in practical applications.

Abstract

State-of-the-art methods for solving smooth optimization problems are nonlinear conjugate gradient, low memory BFGS, and Majorize-Minimize (MM) subspace algorithms. The MM subspace algorithm which has been introduced more recently has shown good practical performance when compared with other methods on various optimization problems arising in signal and image processing. However, to the best of our knowledge, no general result exists concerning the theoretical convergence rate of the MM subspace algorithm. This paper aims at deriving such convergence rates both for batch and online versions of the algorithm and, in particular, discusses the influence of the choice of the subspace.