Bayesian Design of Experiments in the Presence of Nuisance Parameters

TL;DR

This paper introduces a Bayesian methodology using BART and large-sample properties to efficiently determine optimal sample sizes and decision criteria in experimental design, especially when nuisance parameters are present.

Contribution

It presents a fully Bayesian approach that accounts for nuisance parameters during the design phase, reducing computational costs and improving Bayesian design adoption.

Findings

Method effectively incorporates nuisance parameter uncertainty.

Significantly reduces computational burden in Bayesian design.

Enables practical application of Bayesian operating characteristics.

Abstract

Design of experiments has traditionally relied on the frequentist hypothesis testing framework where the optimal size of the experiment is specified as the minimum sample size that guarantees a required level of power. Sample size determination may be performed analytically when the test statistic has a known asymptotic sampling distribution and, therefore, the power function is available in analytic form. Bayesian methods have gained popularity in all stages of discovery, namely, design, analysis and decision making. Bayesian decision procedures rely on posterior summaries whose sampling distributions are commonly estimated via Monte Carlo simulations. In the design of scientific studies, the Bayesian approach incorporates uncertainty about the design value(s) instead of conditioning on a single value of the model parameter(s). Accounting for uncertainties in the design value(s) is particularly critical when the model includes nuisance parameters. In this manuscript, we propose methodology that utilizes the large-sample properties of the posterior distribution together with Bayesian additive regression trees (BART) to efficiently obtain the optimal sample size and decision criteria in fixed and adaptive designs. We introduce a fully Bayesian procedure that incorporates the uncertainty associated with the model parameters including the nuisance parameters at the design stage. The proposed approach significantly reduces the computational burden associated with Bayesian design and enables the wide adoption of Bayesian operating characteristics.