Lp-norms, Log-barriers and Cramer transform in Optimization

TL;DR

This paper explores how Lp-norms, log-barriers, and the Cramer transform relate in optimization, revealing new connections and methods for dual problem formulation through Laplace approximation techniques.

Contribution

It demonstrates that Laplace approximation of supremums via Lp-norms naturally leads to the emergence of logarithmic barrier functions and dual formulations in convex optimization.

Findings

Logarithmic barrier functions appear naturally from Lp-norm approximations.

Dual problems can be explicitly derived using the Cramer transform.

The technique enables dual problem formulation when conjugates are not explicitly known.

Abstract

We show that the Laplace approximation of a supremum by Lp-norms has interesting consequences in optimization. For instance, the logarithmic barrier functions (LBF) of a primal convex problem P and its dual appear naturally when using this simple approximation technique for the value function g of P or its Legendre-Fenchel conjugate. In addition, minimizing the LBF of the dual is just evaluating the Cramer transform of the Laplace approximation of g. Finally, this technique permits to sometimes define an explicit dual problem in cases when the Legendre-Fenchel conjugate of g cannot be derived explicitly from its definition.