Testing and Improving the Robustness of Amortized Bayesian Inference for Cognitive Models

TL;DR

This paper evaluates and enhances the robustness of amortized Bayesian inference for cognitive models, particularly in the presence of outliers, by analyzing sensitivity and proposing a contamination-aware training method.

Contribution

It introduces a data augmentation approach with contamination distributions to improve the robustness of neural estimators in Bayesian inference for cognitive models.

Findings

Contaminant observations significantly affect parameter estimation.

Data augmentation with Cauchy distribution improves robustness.

Proposed method is practical and broadly applicable.

Abstract

Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models, which are statistical models representing cognitive processes. In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks. To this end, we conduct systematic analyses on a toy example and analyze both synthetic and real data using a popular cognitive model, the Drift Diffusion Models (DDM). First, we study the sensitivity of ABI to contaminants with tools from robust statistics: the empirical influence function and the breakdown point. Next, we propose a data augmentation or noise injection approach that incorporates a contamination distribution into the data-generating process during training. We examine several candidate distributions and evaluate their performance and cost in terms of accuracy and efficiency loss relative to a standard estimator. Introducing contaminants from a Cauchy distribution during training considerably increases the robustness of the neural density estimator as measured by bounded influence functions and a much higher breakdown point. Overall, the proposed method is straightforward and practical to implement and has a broad applicability in fields where outlier detection or removal is challenging.