Approximation by log-concave distributions, with applications to regression

TL;DR

This paper investigates how arbitrary distributions can be approximated by log-concave distributions using a KL-divergence approach, establishing existence, continuity, and applications to regression models with log-concave errors.

Contribution

It proves the existence and continuity of the best log-concave approximation for arbitrary distributions and applies these results to establish estimators in regression models with log-concave errors.

Findings

Approximation exists if the distribution has finite first moments and is not supported on a hyperplane.

The approximation depends continuously on the distribution in Mallows distance.

Establishes the consistency of maximum likelihood estimators for log-concave densities and regression models.

Abstract

We study the approximation of arbitrary distributions $P$ on $d$-dimensional space by distributions with log-concave density. Approximation means minimizing a Kullback--Leibler-type functional. We show that such an approximation exists if and only if $P$ has finite first moments and is not supported by some hyperplane. Furthermore we show that this approximation depends continuously on $P$ with respect to Mallows distance $D_1(\cdot,\cdot)$. This result implies consistency of the maximum likelihood estimator of a log-concave density under fairly general conditions. It also allows us to prove existence and consistency of estimators in regression models with a response $Y=\mu(X)+\epsilon$, where $X$ and $\epsilon$ are independent, $\mu(\cdot)$ belongs to a certain class of regression functions while $\epsilon$ is a random error with log-concave density and mean zero.