Obtaining Valid Compatibility Intervals for Sequence Symmetry Analyses Utilizing Active Comparators: A Simulation Study

TL;DR

This study compares methods for calculating compatibility intervals in sequence symmetry analyses and finds the Miettinen–Nurminen score estimator to be the most accurate.

Contribution

The study identifies the Miettinen–Nurminen score estimator as superior for calculating 95% compatibility intervals in sequence symmetry analyses with active comparators.

Findings

The Miettinen–Nurminen score estimator provided the most accurate coverage (0.951) for 95% compatibility intervals.

Current recommendations using exact Clopper–Pearson intervals showed higher coverage (0.979), indicating over-conservatism.

Acceptable coverage was achieved with sample sizes exceeding 15 for most methods except the exact Clopper–Pearson.

Abstract

To compare different methods of estimating 95% compatibility intervals (CIs) for the sequence ratio (SR) when performing a sequence symmetry analysis using an active comparator to reduce the risk of time‐varying confounding.

We conducted a simulation study, where we simulated drug‐outcome and outcome‐drug sequences for a drug of interest and a comparator drug using the binomial distribution and obtained active comparator SRs and 95% CIs. We simulated scenarios with sample sizes between 5 and 50 observed sequences for each SR, which could take values of 0.5, 1.0, or 2.0, yielding 276 scenarios that were replicated 5000 times. For each replication, we calculated 95% CIs using current recommendations based on exact CIs, the Woolf logit, Baptista‐Pike mid‐p, and Miettinen–Nurminen score estimator and calculated coverage for each scenario.

All interval estimators provided acceptable coverage when sample sizes exceeded 15, except for the current recommendation, the exact Clopper–Pearson interval. The Miettinen‐Nurminen score (coverage 0.951) and Baptista–Pike mid‐p interval (coverage 0.955) offered more accurate coverage than other methods. The largest divergence from 0.95 was observed for the current recommendations (coverage 0.979).

The Miettinen–Nurminen score estimator provided the most accurate coverage for 95% CIs of active comparator SRs, especially with low sample sizes. Therefore, we recommend using the Miettinen–Nurminen score estimator for active comparator SRs.

Figures