Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance

TL;DR

This paper compares Kullback--Leibler divergence and Wasserstein distance as utility functions in Bayesian experimental design, revealing their respective advantages and limitations in model discrepancy calibration.

Contribution

It systematically analyzes the trade-offs between KL divergence and Wasserstein distance in Bayesian experimental design, providing practical guidelines for their use.

Findings

KL divergence leads to faster convergence without model discrepancy

Wasserstein distance offers more robustness when model discrepancy exists

Wasserstein can exhibit false rewards unrelated to information gain

Abstract

Designing experiments that systematically gather data from complex physical systems is central to accelerating scientific discovery. While Bayesian experimental design (BED) provides a principled, information-based framework that integrates experimental planning with probabilistic inference, the selection of utility functions in BED is a long-standing and active topic, where different criteria emphasize different notions of information. Although Kullback--Leibler (KL) divergence has been one of the most common choices, recent studies have proposed Wasserstein distance as an alternative. In this work, we first employ a toy example to illustrate an issue of Wasserstein distance - the value of Wasserstein distance of a fixed-shape posterior depends on the relative position of its main mass within the support and can exhibit false rewards unrelated to information gain, especially with a non-informative prior (e.g., uniform distribution). We then further provide a systematic comparison between these two criteria through a classical source inversion problem in the BED literature, revealing that the KL divergence tends to lead to faster convergence in the absence of model discrepancy, while Wasserstein metrics provide more robust sequential BED results if model discrepancy is non-negligible. These findings clarify the trade-offs between KL divergence and Wasserstein metrics for the utility function and provide guidelines for selecting suitable criteria in practical BED applications.