Hierarchical Bayesian Regression for experimental psychology: a case study of cognitive control

TL;DR

This paper explores how Hierarchical Bayesian Regression improves statistical analysis in experimental psychology, using datasets from a cognitive control task.

Contribution

The paper introduces a novel application of Hierarchical Bayesian Regression to model cognitive control effects with cumulative posteriors and detailed response patterns.

Findings

Hierarchical Bayesian Regression provides cumulative posterior distributions for effects of interest.

The method estimates consistency of effects and strength of null effects across datasets.

It accurately models response time distributions and trial-level accuracy patterns.

Abstract

Arising from the so-called ‘replication crisis’ in the experimental psychology literature, there has been a growing call to reassess whether specific analytic practices might enhance the accuracy and precision of reported findings. This issue is explored here, through a case study examination of two previously collected datasets from the Dual Mechanisms of Cognitive Control (DMCC) task battery. This case study highlights the unique advantages afforded by Hierarchical Bayesian Regression (HBR) models as a potentially more rigorous analytic approach to statistical inference. In the DMCC datasets, two sets of HBR models are presented, with the estimates of the former used as priors for the latter. In addition to systematically generating cumulative posterior distributions for all effects of theoretical interest, we further illustrate how our particular application of HBR models provides novel insights regarding specific indicators of proactive/reactive control in each of the four DMCC tasks, by: (1) estimating the consistency of effects across datasets; (2) estimating the relative strength of null effects; (3) accurately modeling the specific properties of response time distributions; and (4) appropriately modeling accuracy patterns at the trial level.

Figures