Forward selection and post-selection inference in factorial designs

TL;DR

This paper develops a rigorous statistical framework for forward selection in factorial designs, establishing consistency and inference methods based on the potential outcome framework without relying on outcome models.

Contribution

It provides the first formal design-based theory for forward factorial effect selection and post-selection inference, including consistency proofs and strategies for higher-order interactions.

Findings

Proves consistency of forward selection in factorial designs.

Quantifies efficiency gains in effect estimation post-selection.

Proposes strategies for inference with higher-order interactions.

Abstract

Ever since the seminal work of R. A. Fisher and F. Yates, factorial designs have been an important experimental tool to simultaneously estimate the effects of multiple treatment factors. In factorial designs, the number of treatment combinations grows exponentially with the number of treatment factors, which motivates the forward selection strategy based on the sparsity, hierarchy, and heredity principles for factorial effects. Although this strategy is intuitive and has been widely used in practice, its rigorous statistical theory has not been formally established. To fill this gap, we establish design-based theory for forward factor selection in factorial designs based on the potential outcome framework. We not only prove a consistency property for the factor selection procedure but also discuss statistical inference after factor selection. In particular, with selection consistency, we quantify the advantages of forward selection based on asymptotic efficiency gain in estimating factorial effects. With inconsistent selection in higher-order interactions, we propose two strategies and investigate their impact on subsequent inference. Our formulation differs from the existing literature on variable selection and post-selection inference because our theory is based solely on the physical randomization of the factorial design and does not rely on a correctly specified outcome model.