Effect of a preliminary test of homogeneity of stratum-specific odds ratios on their confidence intervals

TL;DR

This paper investigates how conducting a preliminary homogeneity test of stratum-specific odds ratios affects the accuracy of confidence intervals in case-control studies, revealing significant negative impacts on coverage probabilities.

Contribution

It provides a detailed analysis of the statistical properties of this two-stage approach, including Monte Carlo and large-sample methods, highlighting potential issues in practical applications.

Findings

Preliminary homogeneity tests can severely reduce confidence interval coverage.

Large-sample and Monte Carlo methods effectively evaluate the statistical properties.

Real-life data analysis confirms the negative impact on coverage probabilities.

Abstract

Consider a case-control study in which the aim is to assess the effect of a factor on disease occurrence. We suppose that this factor is dichotomous. Also suppose that the data consists of two strata, each stratum summarized by a two-by-two table. A commonly-proposed two-stage analysis of this type of data is the following. We carry out a preliminary test of homogeneity of the stratum-specific odds ratios. If the null hypothesis of homogeneity is accepted then we find a confidence interval for the assumed common value (across strata) of the odds ratio. We examine the statistical properties of this two-stage analysis, based on the Woolf method, on confidence intervals constructed for the stratum-specific odds ratios, for large numbers of cases and controls for each stratum. We provide both a Monte Carlo simulation method and an elegant large-sample method for this examination. These methods are applied to obtain numerical results in the context of the large numbers of cases and controls for each stratum that arose in a real-life dataset. In this context, we find that the preliminary test of homogeneity of the stratum-specific odds ratios has a very harmful effect on the coverage probabilities of these confidence intervals.