Rejoinder to Reader Reaction "On exact randomization-based covariate-adjusted confidence intervals" by Jacob Fiksel

TL;DR

This paper develops an analytical method to invert Fisher randomization tests for non-monotonic p-value functions, enabling accurate covariate-adjusted confidence intervals in randomized experiments.

Contribution

It introduces a new analytical approach to invert FRTs for non-monotonic p-value functions, including the studentized t-statistic, improving confidence interval accuracy.

Findings

The method guarantees desired coverage probability.

Simulation results show computational efficiency.

Resolves contradictions in previous studies.

Abstract

We applaud Fiksel (2024) for their valuable contributions to randomization-based inference, particularly their work on inverting the Fisher randomization test (FRT) to construct confidence intervals using the covariate-adjusted test statistic. FRT is advocated by many scholars because it produces finite-sample exact p-values for any test statistic and can be easily adopted for any experimental design (Rosenberger et al., 2019; Proschan and Dodd, 2019; Young, 2019; Bind and Rubin, 2020). By inverting FRTs, we can construct the randomization-based confidence interval (RBCI). To the best of our knowledge, Zhu and Liu (2023) are the first to analytically invert the FRT for the difference-in-means statistic. Fiksel (2024) extended this analytical approach to the covariate-adjusted statistic, producing a monotonic p-value function under certain conditions. In this rejoinder, we propose an analytical approach to invert the FRT for test statistics that yield a non-monotonic p-value function, with the studentized t-statistic as an important special case. Exploiting our analytical approach, we can recover the non-monotonic p-value function and construct RBCI based on the studentized t-statistic. The RBCI generated by the proposed analytical approach is guaranteed to achieve the desired coverage probability and resolve the contradiction between Luo et al. (2021) and Wu and Ding (2021). Simulation results validate our findings and demonstrate that our method is also computationally efficient.