EM algorithms for optimization problems with polynomial objectives

TL;DR

This paper extends the EM algorithm framework to optimize polynomial objectives over polyhedral sets by employing exponential family distributions, demonstrating convergence and providing practical examples.

Contribution

It introduces a novel approach to apply EM algorithms to non-statistical polynomial optimization problems with convergence analysis.

Findings

EM can be applied to polynomial optimization with exponential family distributions.

Global convergence is established for specific cases with convex quadratic objectives.

The approach is demonstrated through three practical examples.

Abstract

The EM (Expectation-Maximization) algorithm is regarded as an MM (Majorization-Minimization) algorithm for maximum likelihood estimation of statistical models. Expanding this view, this paper demonstrates that by choosing an appropriate probability distribution, even nonstatistical optimization problem can be cast as a negative log-likelihood-like minimization problem, which can be approached by an EM (or MM) algorithm. When a polynomial objective is optimized over a simple polyhedral feasible set and an exponential family distribution is employed, the EM algorithm can be reduced to a natural gradient descent of the employed distribution with a constant step size. This is demonstrated through three examples. In this paper, we demonstrate the global convergence of specific cases with some exponential family distributions in a general form. In instances when the feasible set is not sufficiently simple, the use of MM algorithms can nevertheless be adequately described. When the objective is to minimize a convex quadratic function and the constraints are polyhedral, global convergence can also be established based on the existing results for an entropy-like proximal point algorithm.