Optimal experimental design: Formulations and computations

TL;DR

This paper surveys modern optimal experimental design (OED), covering its theoretical foundations, computational challenges, and emerging sequential methods, with a focus on complex models and adaptive strategies.

Contribution

It provides a comprehensive overview of OED formulations, estimation techniques, optimization methods, and recent advances in sequential design, highlighting open challenges.

Findings

Bayesian and decision-theoretic criteria effectively encode experimental goals.

Estimating design criteria is challenging due to nonlinearities and high dimensions.

Sequential OED methods adapt experiments based on previous outcomes.

Abstract

Questions of `how best to acquire data' are essential to modeling and prediction in the natural and social sciences, engineering applications, and beyond. Optimal experimental design (OED) formalizes these questions and creates computational methods to answer them. This article presents a systematic survey of modern OED, from its foundations in classical design theory to current research involving OED for complex models. We begin by reviewing criteria used to formulate an OED problem and thus to encode the goal of performing an experiment. We emphasize the flexibility of the Bayesian and decision-theoretic approach, which encompasses information-based criteria that are well-suited to nonlinear and non-Gaussian statistical models. We then discuss methods for estimating or bounding the values of these design criteria; this endeavor can be quite challenging due to strong nonlinearities, high parameter dimension, large per-sample costs, or settings where the model is implicit. A complementary set of computational issues involves optimization methods used to find a design; we discuss such methods in the discrete (combinatorial) setting of observation selection and in settings where an exact design can be continuously parameterized. Finally we present emerging methods for sequential OED that build non-myopic design policies, rather than explicit designs; these methods naturally adapt to the outcomes of past experiments in proposing new experiments, while seeking coordination among all experiments to be performed. Throughout, we highlight important open questions and challenges.