A Bayes Factor for Replications of ANOVA Results

TL;DR

This paper extends the Replication Bayes Factor, originally for t-tests, to ANOVA designs, providing a Bayesian method to evaluate replication success in psychological studies.

Contribution

The paper introduces a Bayesian Replication Bayes Factor for ANOVA, expanding its application beyond t-tests to multi-group fixed-effect designs.

Findings

Demonstrates the usefulness of the extended Bayes Factor through simulations and examples.

Compares the new approach to other Bayesian and frequentist methods.

Provides R code for calculating Replication Bayes Factors.

Abstract

With an increasing number of replication studies performed in psychological science, the question of how to evaluate the outcome of a replication attempt deserves careful consideration. Bayesian approaches allow to incorporate uncertainty and prior information into the analysis of the replication attempt by their design. The Replication Bayes Factor, introduced by Verhagen & Wagenmakers (2014), provides quantitative, relative evidence in favor or against a successful replication. In previous work by Verhagen & Wagenmakers (2014) it was limited to the case of $t$-tests. In this paper, the Replication Bayes Factor is extended to $F$-tests in multi-group, fixed-effect ANOVA designs. Simulations and examples are presented to facilitate the understanding and to demonstrate the usefulness of this approach. Finally, the Replication Bayes Factor is compared to other Bayesian and frequentist approaches and discussed in the context of replication attempts. R code to calculate Replication Bayes factors and to reproduce the examples in the paper is available at https://osf.io/jv39h/.