Fitting High-Dimensional Interaction Models with Error Control

TL;DR

This paper introduces RAI, a sequential algorithm for high-dimensional polynomial regression that effectively identifies influential interactions while controlling false discoveries and ensuring model generalization.

Contribution

The paper proposes RAI, a novel greedy algorithm with performance guarantees for selecting interactions in high-dimensional models, especially in gene expression analysis.

Findings

RAI effectively controls false rejections.

RAI provides performance guarantees relative to the best model.

Application to gene-specific interaction models demonstrates practical utility.

Abstract

There is a renewed interest in polynomial regression in the form of identifying influential interactions between features. In many settings, this takes place in a high-dimensional model, making the number of interactions unwieldy or computationally infeasible. Furthermore, it is difficult to analyze such spaces directly as they are often highly correlated. Standard feature selection issues remain such as how to determine a final model which generalizes well. This paper solves these problems with a sequential algorithm called Revisiting Alpha-Investing (RAI). RAI is motivated by the principle of marginality and searches the feature-space of higher-order interactions by greedily building upon lower-order terms. RAI controls a notion of false rejections and comes with a performance guarantee relative to the best-subset model. This ensures that signal is identified while providing a valid stopping criterion to prevent over-selection. We apply RAI in a novel setting over a family of regressions in order to select gene-specific interaction models for differential expression profiling.