Maximum Entropy on the Mean and the Cram\'er Rate Function in Statistical Estimation and Inverse Problems: Properties, Models, and Algorithms

TL;DR

This paper investigates the properties and algorithms of the Maximum Entropy on the Mean (MEM) method, linking it to the Cramér rate function, and demonstrates its application in regularized inverse problems with an efficient computational framework.

Contribution

It provides a theoretical analysis of the MEM function and Cramér rate function, and introduces a new algorithmic framework for solving MEM-based inverse problems.

Findings

Conditions for the Cramér rate function representation are established.

A Bregman proximal gradient algorithm for MEM-based models is proposed.

A software package for MEM applications is provided.

Abstract

We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core of this approach lies the MEM function which is a partial minimization of the Kullback-Leibler divergence over a linear constraint. In many cases, it is known that this function admits a simpler representation (known as the Cram\'er rate function). Via the connection to exponential families of probability distributions, we study general conditions under which this representation holds. We then address how the associated MEM estimator gives rise to a wide class of MEM-based regularized linear models for solving inverse problems. Finally, we propose an algorithmic framework to solve these problems efficiently based on the Bregman proximal gradient method, alongside proximal operators for commonly used reference distributions. The article is complemented by a software package for experimentation and exploration of the MEM approach in applications.