Generalized Median of Means Principle for Bayesian Inference

TL;DR

This paper extends the median of means estimator to Bayesian inference, creating a robust posterior that is resistant to outliers and maintains statistical guarantees, with promising applications.

Contribution

It introduces the robust posterior distribution in Bayesian inference, quantifies its robustness, and establishes theoretical guarantees similar to classical Bayesian results.

Findings

Robust posterior is resistant to outliers.

The approach satisfies a Bernstein-von Mises type theorem.

Performs well in practical applications.

Abstract

The topic of robustness is experiencing a resurgence of interest in the statistical and machine learning communities. In particular, robust algorithms making use of the so-called median of means estimator were shown to satisfy strong performance guarantees for many problems, including estimation of the mean, covariance structure as well as linear regression. In this work, we propose an extension of the median of means principle to the Bayesian framework, leading to the notion of the robust posterior distribution. In particular, we (a) quantify robustness of this posterior to outliers, (b) show that it satisfies a version of the Bernstein-von Mises theorem that connects Bayesian credible sets to the traditional confidence intervals, and (c) demonstrate that our approach performs well in applications.