A Better (Bayesian) Interval Estimate for Within-Subject Designs

TL;DR

This paper introduces a Bayesian highest-density interval for within-subject designs that improves upon traditional confidence intervals by being more sensible, interpretable, and consistently shorter, especially for heteroscedastic data.

Contribution

It develops a Bayesian within-subject HDI based on a modified posterior, offering a superior alternative to existing confidence intervals and providing a Bayesian perspective on normalization methods.

Findings

The new interval is always shorter than the traditional within-subject confidence interval.

It can be generalized to heteroscedastic data, matching the normalization method.

The proposed interval has a clear Bayesian interpretation and is based on a more sensible prior.

Abstract

We develop a Bayesian highest-density interval (HDI) for use in within-subject designs. This credible interval is based on a standard noninformative prior and a modified posterior distribution that conditions on both the data and point estimates of the subject-specific random effects. Conditioning on the estimated random effects removes between-subject variance and produces intervals that are the Bayesian analogue of the within-subject confidence interval proposed in Loftus and Masson (1994). We show that the latter interval can also be derived as a Bayesian within-subject HDI under a certain improper prior. We argue that the proposed new interval is superior to the original within-subject confidence interval, on the grounds of (a) it being based on a more sensible prior, (b) it having a clear and intuitively appealing interpretation, and (c) because its length is always smaller. A generalization of the new interval that can be applied to heteroscedastic data is also derived, and we show that the resulting interval is numerically equivalent to the normalization method discussed in Franz and Loftus (2012); however, our work provides a Bayesian formulation for the normalization method, and in doing so we identify the associated prior distribution.