A systematic optimization approach for a class of statistical inference problems utilizing data augmentation

TL;DR

This paper introduces a systematic optimization algorithm for statistical inference problems involving hidden variables, combining ideas from MM and EM algorithms to improve surrogate function construction.

Contribution

It proposes a unified, iterative optimization framework for various inference problems with hidden variables, enhancing existing methods with a systematic surrogate function approach.

Findings

Effective in handling hidden variables in ML and MAP estimation

Applicable to Instrumental Variables and regularized problems

Numerical examples demonstrate improved performance

Abstract

We present an algorithm for a class of statistical inference problems. The main idea is to reformulate the inference problem as an optimization procedure, based on the generation of surrogate (auxiliary) functions. This approach is motivated by the MM algorithm, combined with the systematic and iterative structure of the Expectation-Maximization algorithm. The resulting algorithm can deal with hidden variables in Maximum Likelihood and Maximum a Posteriori estimation problems, Instrumental Variables, Regularized Optimization and Constrained Optimization problems. The advantage of the proposed algorithm is to provide a systematic procedure to build surrogate functions for a class of problems where hidden variables are usually involved. Numerical examples show the benefits of the proposed approach.