Familywise Error Rate Controlling Procedures for Discrete Data

TL;DR

This paper introduces three new stepwise procedures for controlling the family-wise error rate in discrete data, which are more powerful than existing methods and applicable in clinical safety analysis.

Contribution

The paper develops three modified FWER controlling procedures for discrete data that outperform existing Tarone-type methods and are applicable under arbitrary dependence.

Findings

Proposed procedures strongly control FWER in various dependence settings.

New methods demonstrate superior power in simulations.

Application to clinical data illustrates practical utility.

Abstract

In applications such as clinical safety analysis, the data of the experiments usually consists of frequency counts. In the analysis of such data, researchers often face the problem of multiple testing based on discrete test statistics, aimed at controlling family-wise error rate (FWER). Most existing FWER controlling procedures are developed for continuous data, which are often conservative when analyzing discrete data. By using minimal attainable $p$-values, several FWER controlling procedures have been specifically developed for discrete data in the literature. In this paper, by utilizing known marginal distributions of true null $p$-values, three more powerful stepwise procedures are developed, which are modified versions of the conventional Bonferroni, Holm and Hochberg procedures, respectively. It is shown that the first two procedures strongly control the FWER under arbitrary dependence and are more powerful than the existing Tarone-type procedures, while the last one only ensures control of the FWER in special settings. Through extensive simulation studies, we provide numerical evidence of superior performance of the proposed procedures in terms of the FWER control and minimal power. A real clinical safety data is used to demonstrate applications of our proposed procedures. An R package "MHTdiscrete" and a web application are developed for implementing the proposed procedures.