Interaction Pursuit with Feature Screening and Selection

TL;DR

This paper introduces the interaction pursuit (IP) method for efficient interaction identification in ultra-high dimensional data, combining feature screening and regularization to improve accuracy and effectiveness.

Contribution

It proposes a novel two-step approach that screens interactions separately from main effects, with theoretical guarantees and practical validation.

Findings

The method achieves sure screening property for interactions and main effects.

It demonstrates superior performance over existing methods in simulations.

The approach is validated with real data examples.

Abstract

Understanding how features interact with each other is of paramount importance in many scientific discoveries and contemporary applications. Yet interaction identification becomes challenging even for a moderate number of covariates. In this paper, we suggest an efficient and flexible procedure, called the interaction pursuit (IP), for interaction identification in ultra-high dimensions. The suggested method first reduces the number of interactions and main effects to a moderate scale by a new feature screening approach, and then selects important interactions and main effects in the reduced feature space using regularization methods. Compared to existing approaches, our method screens interactions separately from main effects and thus can be more effective in interaction screening. Under a fairly general framework, we establish that for both interactions and main effects, the method enjoys the sure screening property in screening and oracle inequalities in selection. Our method and theoretical results are supported by several simulation and real data examples.