A reinforced learning approach to optimal design under model uncertainty

TL;DR

This paper introduces a reinforced learning sequential algorithm for optimal experimental design that remains efficient in parameter estimation and model discrimination, even under model uncertainty.

Contribution

It develops a novel reinforced learning approach for designing experiments that adaptively optimize for model selection and parameter estimation under uncertainty.

Findings

The algorithm achieves asymptotic efficiency comparable to the true model's optimal design.

Finite-stage designs have a quantifiable lower bound on relative efficiency.

The method effectively discriminates between candidate models in simulations.

Abstract

Optimal designs are usually model-dependent and likely to be sub-optimal if the postulated model is not correctly specified. In practice, it is common that a researcher has a list of candidate models at hand and a design has to be found that is efficient for selecting the true model among the competing candidates and is also efficient (optimal, if possible) for estimating the parameters of the true model. In this article, we use a reinforced learning approach to address this problem. We develop a sequential algorithm, which generates a sequence of designs which have asymptotically, as the number of stages increases, the same efficiency for estimating the parameters in the true model as an optimal design if the true model would have correctly been specified in advance. A lower bound is established to quantify the relative efficiency between such a design and an optimal design for the true model in finite stages. Moreover, the resulting designs are also efficient for discriminating between the true model and other rival models from the candidate list. Some connections with other state-of-the-art algorithms for model discrimination and parameter estimation are discussed and the methodology is illustrated by a small simulation study.