An Elementary Proof of the Near Optimality of LogSumExp Smoothing

TL;DR

This paper proves that the LogSumExp smoothing of the max function in high dimensions is nearly optimal, establishing a lower bound that matches its overestimation up to small constants, and provides exact solutions in low dimensions.

Contribution

It introduces an elementary lower bound for smoothing max functions and demonstrates the near optimality of LogSumExp, with exact solutions in small dimensions.

Findings

LogSumExp differs from the max function by at most ln(d).

Any smoothing must overestimate by at least ~0.8145 ln(d).

LogSumExp is nearly optimal up to small constants.

Abstract

We consider the design of smoothings of the (coordinate-wise) max function in $\mathbb{R}^d$ in the infinity norm. The LogSumExp function $f(x)=\ln(\sum^d_i\exp(x_i))$ provides a classical smoothing, differing from the max function in value by at most $\ln(d)$. We provide an elementary construction of a lower bound, establishing that every overestimating smoothing of the max function must differ by at least $\sim 0.8145\ln(d)$. Hence, LogSumExp is optimal up to small constant factors. However, in small dimensions, we provide stronger, exactly optimal smoothings attaining our lower bound, showing that the entropy-based LogSumExp approach to smoothing is not exactly optimal.