Posterior concentration and fast convergence rates for generalized Bayesian learning

TL;DR

This paper establishes conditions under which generalized Bayesian methods achieve fast convergence rates, even with complex hypothesis spaces and heavy-tailed loss functions, demonstrating robustness in Bayesian linear regression.

Contribution

It proves posterior concentration and fast learning rates for generalized Bayes estimators in complex, irregular settings, extending understanding of Bayesian robustness.

Findings

Posterior concentrates around optimal hypotheses under Bernstein's condition.

Generalized Bayes estimator achieves fast learning rates.

Bayesian linear regression is robust to heavy-tailed distributions.

Abstract

In this paper, we study the learning rate of generalized Bayes estimators in a general setting where the hypothesis class can be uncountable and have an irregular shape, the loss function can have heavy tails, and the optimal hypothesis may not be unique. We prove that under the multi-scale Bernstein's condition, the generalized posterior distribution concentrates around the set of optimal hypotheses and the generalized Bayes estimator can achieve fast learning rate. Our results are applied to show that the standard Bayesian linear regression is robust to heavy-tailed distributions.