Bayesian Optimal Experimental Design for Simulator Models of Cognition

TL;DR

This paper introduces a novel Bayesian optimal experimental design approach for complex, intractable simulator models of cognition, leveraging machine learning for improved experiment selection and inference.

Contribution

It combines recent advances in BOED and approximate inference with machine learning to handle intractable models in cognitive science.

Findings

Enhanced model discrimination in simulations

Improved parameter estimation accuracy

Outperforms traditional experimental designs

Abstract

Bayesian optimal experimental design (BOED) is a methodology to identify experiments that are expected to yield informative data. Recent work in cognitive science considered BOED for computational models of human behavior with tractable and known likelihood functions. However, tractability often comes at the cost of realism; simulator models that can capture the richness of human behavior are often intractable. In this work, we combine recent advances in BOED and approximate inference for intractable models, using machine-learning methods to find optimal experimental designs, approximate sufficient summary statistics and amortized posterior distributions. Our simulation experiments on multi-armed bandit tasks show that our method results in improved model discrimination and parameter estimation, as compared to experimental designs commonly used in the literature.