Clusterization in D-optimal designs: the case against linearization

TL;DR

This paper investigates the tendency of D-optimal designs to produce clustered measurement locations in linear inverse problems, showing that correlation among errors can reduce clustering and challenging the use of linearized models with Gaussian priors.

Contribution

It provides a theoretical framework for understanding clusterization in D-optimal designs and proposes that error correlation mitigates this issue, offering new insights into optimal experimental design.

Findings

Clusterization is a common trait of D-optimal designs for linear inverse problems.

Introducing correlations among measurement errors reduces clusterization.

Measurement clusterization results from the pigeonhole principle when measurements exceed eigenvector variations.

Abstract

Estimation of parameters in physical processes often demands costly measurements, prompting the pursuit of an optimal measurement strategy. Finding such strategy is termed the problem of optimal experimental design, abbreviated as optimal design. Remarkably, optimal designs can yield tightly clustered measurement locations, leading researchers to fundamentally revise the design problem just to circumvent this issue. Some authors introduce error correlation among error terms that are initially independent, while others restrict measurement locations to a finite set of locations. While both approaches may prevent clusterization, they also fundamentally alter the optimal design problem. In this study, we consider Bayesian D-optimal designs, i.e.~designs that maximize the expected Kullback-Leibler divergence between posterior and prior. We propose an analytically tractable model for D-optimal designs over Hilbert spaces. In this framework, we make several key contributions: (a) We establish that measurement clusterization is a generic trait of D-optimal designs for linear inverse problems with independent Gaussian measurement errors and a Gaussian prior. (b) We prove that introducing correlations among measurement error terms mitigates clusterization. (c) We characterize D-optimal designs as reducing uncertainty across a subset of prior covariance eigenvectors. (d) We leverage this characterization to argue that measurement clusterization arises as a consequence of the pigeonhole principle: when more measurements are taken than there are locations where the select eigenvectors are large and others are small -- clusterization occurs. Finally, we use our analysis to argue against the use of Gaussian priors with linearized physical models when seeking a D-optimal design.