Maximin Robust Bayesian Experimental Design

TL;DR

This paper introduces a robust Bayesian experimental design framework using a max-min game approach that leverages Sibson's alpha-mutual information to improve resilience against model misspecification.

Contribution

It formulates a novel max-min game for robust design, linking Sibson's alpha-MI with Rènyi divergence, and employs a PAC-Bayes approach to optimize stochastic policies with finite-sample guarantees.

Findings

The approach yields a robust objective based on Sibson's alpha-MI.

The Rènyi divergence is identified as the key measure of information gain.

A PAC-Bayes framework provides high-probability bounds on the estimated information gain.

Abstract

We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $\alpha$-mutual information (MI), which identifies the $\alpha$-tilted posterior as the robust belief update and establishes the R\'enyi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $\alpha$-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.