Weighted Particle-Based Optimization for Efficient Generalized Posterior Calibration

TL;DR

This paper introduces an efficient weighted particle-based method to calibrate generalized posteriors in Bayesian inference, significantly reducing computational costs while maintaining accurate coverage probabilities.

Contribution

It develops a novel SMC-inspired algorithm for generalized posterior calibration that avoids repeated posterior simulations, improving efficiency over existing methods.

Findings

Reduces calibration computational cost by using reweighting instead of repeated simulations.

Achieves accurate coverage probability calibration with fewer iterations.

Demonstrates effectiveness on synthetic and real datasets.

Abstract

In the realm of statistical learning, the increasing volume of accessible data and increasing model complexity necessitate robust methodologies. This paper explores two branches of robust Bayesian methods in response to this trend. The first is generalized Bayesian inference, which introduces a learning rate parameter to enhance robustness against model misspecifications. The second is Gibbs posterior inference, which formulates inferential problems using generic loss functions rather than probabilistic models. In such approaches, it is necessary to calibrate the spread of the posterior distribution by selecting a learning rate parameter. The study aims to enhance the generalized posterior calibration (GPC) algorithm proposed by [1]. Their algorithm chooses the learning rate to achieve the nominal frequentist coverage probability, but it is computationally intensive because it requires repeated posterior simulations for bootstrap samples. We propose a more efficient version of the GPC inspired by sequential Monte Carlo (SMC) samplers. A target distribution with a different learning rate is evaluated without posterior simulation as in the reweighting step in SMC sampling. Thus, the proposed algorithm can reach the desirable value within a few iterations. This improvement substantially reduces the computational cost of the GPC. Its efficacy is demonstrated through synthetic and real data applications.