Alternating Minimization, Proximal Minimization and Optimization Transfer Are Equivalent

TL;DR

This paper demonstrates the equivalence of proximal minimization, majorization minimization, and alternating minimization algorithms, providing new convergence conditions and insights into their relationships.

Contribution

It establishes the theoretical equivalence among PMA, MM, and AM algorithms and introduces new convergence conditions for these methods.

Findings

Proximal minimization, MM, and AM are mathematically equivalent.

New conditions ensure the convergence of these algorithms to the infimum.

Examples illustrate the application of the equivalence and convergence conditions.

Abstract

We show that proximal minimization algorithms (PMA), majorization minimization (MM), and alternating minimization (AM) are equivalent. Each type of algorithm leads to a decreasing sequence of objective function. New conditions on PMA are given (the limit of the decreasing sequence of objective function is indeed the infimum of the objective function), which lead to new conditions on AM for the sequence Phi to converge to its infimum. These conditions can then be translated into the language of MM. Examples are given of each type of algorithm and some open questions are posed.