Connecting Max-entropy With Computational Geometry, LP And SDP

TL;DR

This paper establishes a deep connection between max-entropy problems, computational geometry, linear programming, and semidefinite programming, revealing new dualities and convergence properties.

Contribution

It demonstrates that max-entropy problems relate to computational geometry and can be used to analyze LP and SDP dual formulations through perspective functions.

Findings

Max-entropy is the Cramér transform of a function solving a computational geometry problem.

The perspective function of the max-entropy problem equals the log-barrier of the dual LP.

Analogous results are shown for SDP, linking max-entropy with semidefinite programming.

Abstract

We consider the well-known max-(relative) entropy problem $\Theta$(y) = infQ$\ll$P DKL(Q P ) with Kullback-Leibler divergence on a domain $\Omega$ $\subset$ R d , and with ''moment'' constraints h dQ = y, y $\in$ R m . We show that when m $\le$ d, $\Theta$ is the Cram{\'e}r transform of a function v that solves a simply related computational geometry problem. Also, and remarkably, to the canonical LP: min x$\ge$0 {c T x\,: A x = y}, with A $\in$ R mxd , one may associate a max-entropy problem with a suitably chosen reference measure P on R d + and linear mapping h(x) = Ax, such that its associated perspective function $\epsilon$ $\Theta$(y/$\epsilon$) is the optimal value of the log-barrier formulation (with parameter $\epsilon$) of the dual LP (and so it converges to the LP optimal value as $\epsilon$ $\rightarrow$ 0). An analogous result also holds for the canonical SDP: min X 0 { C, X\,: A(X) = y }.