Characterizing the robustness of Bayesian adaptive experimental designs to active learning bias

TL;DR

This paper investigates how Bayesian adaptive experimental designs, a form of active learning, can suffer from bias due to model misspecification, and offers insights into mitigating this bias through model class choices.

Contribution

It analyzes active learning bias in Bayesian adaptive designs, especially under model misspecification, and demonstrates how model noise levels influence bias severity.

Findings

Worse model misspecification leads to increased active learning bias.

Higher inherent noise in models reduces active learning bias.

Linear model insights can predict bias in nonlinear preference learning experiments.

Abstract

Bayesian adaptive experimental design is a form of active learning, which chooses samples to maximize the information they give about uncertain parameters. Prior work has shown that other forms of active learning can suffer from active learning bias, where unrepresentative sampling leads to inconsistent parameter estimates. We show that active learning bias can also afflict Bayesian adaptive experimental design, depending on model misspecification. We analyze the case of estimating a linear model, and show that worse misspecification implies more severe active learning bias. At the same time, model classes incorporating more "noise" - i.e., specifying higher inherent variance in observations - suffer less from active learning bias. Finally, we demonstrate empirically that insights from the linear model can predict the presence and degree of active learning bias in nonlinear contexts, namely in a (simulated) preference learning experiment.