Robust Bayes-Like Estimation: Rho-Bayes estimation

TL;DR

This paper introduces a robust Bayesian density estimation method that maintains concentration properties even under model misspecification, providing a reliable alternative to classical Bayesian approaches.

Contribution

The paper develops a robust Bayes-like posterior that retains concentration properties under misspecification, unlike traditional Bayesian methods.

Findings

Robust posterior concentrates around the true density under correct model.

Maintains concentration properties under model misspecification.

Hellinger distance between classical and robust posterior tends to zero with more data.

Abstract

We consider the problem of estimating the joint distribution $P$ of $n$ independent random variables within the Bayes paradigm from a non-asymptotic point of view. Assuming that $P$ admits some density $s$ with respect to a given reference measure, we consider a density model $\overline S$ for $s$ that we endow with a prior distribution $\pi$ (with support $\overline S$) and we build a robust alternative to the classical Bayes posterior distribution which possesses similar concentration properties around $s$ whenever it belongs to the model $\overline S$. Furthermore, in density estimation, the Hellinger distance between the classical and the robust posterior distributions tends to 0, as the number of observations tends to infinity, under suitable assumptions on the model and the prior, provided that the model $\overline S$ contains the true density $s$. However, unlike what happens with the classical Bayes posterior distribution, we show that the concentration properties of this new posterior distribution are still preserved in the case of a misspecification of the model, that is when $s$ does not belong to $\overline S$ but is close enough to it with respect to the Hellinger distance.