Comparison of Bayesian and Frequentist Multiplicity Correction For Testing Mutually Exclusive Hypotheses Under Data Dependence

TL;DR

This paper compares Bayesian and frequentist methods for controlling multiple hypothesis tests with dependent data, showing Bayesian approaches often outperform traditional methods in dependent scenarios.

Contribution

It provides a comparative analysis of Bayesian and frequentist multiplicity correction methods under data dependence, highlighting Bayesian methods' effectiveness.

Findings

Bayesian methods exhibit strong frequentist properties.

Bayesian approaches effectively control multiplicity with dependent test statistics.

Bayesian methods maintain power under dependence.

Abstract

The problem of testing mutually exclusive hypotheses with dependent test statistics is considered. Bayesian and frequentist approaches to multiplicity control are studied and compared to help gain understanding as to the effect of test statistic dependence on each approach. The Bayesian approach is shown to have excellent frequentist properties and is argued to be the most effective way of obtaining frequentist multiplicity control, without sacrificing power, when there is considerable test statistic dependence.