Metrics for Bayesian Optimal Experiment Design under Model Misspecification

TL;DR

This paper extends Bayesian optimal experiment design by introducing new information gain metrics that account for model misspecification, enhancing robustness and discrepancy detection in practical scenarios.

Contribution

It proposes an expanded framework with new utility functions for Bayesian experiment design that address model discrepancy and robustness, demonstrated on engineering models.

Findings

New metrics improve robustness to model misspecification.

Framework effectively detects model discrepancies.

Application to engineering models shows practical benefits.

Abstract

The conventional approach to Bayesian decision-theoretic experiment design involves searching over possible experiments to select a design that maximizes the expected value of a specified utility function. The expectation is over the joint distribution of all unknown variables implied by the statistical model that will be used to analyze the collected data. The utility function defines the objective of the experiment where a common utility function is the information gain. This article introduces an expanded framework for this process, where we go beyond the traditional Expected Information Gain criteria and introduce the Expected General Information Gain which measures robustness to the model discrepancy and Expected Discriminatory Information as a criterion to quantify how well an experiment can detect model discrepancy. The functionality of the framework is showcased through its application to a scenario involving a linearized spring mass damper system and an F-16 model where the model discrepancy is taken into account while doing Bayesian optimal experiment design.