Selection of Ultrahigh-Dimensional Interactions Using $L_0$ Penalty

TL;DR

This paper introduces SHL0, a novel method for selecting interactions in ultrahigh-dimensional models that satisfies hierarchical restrictions and outperforms existing methods in simulations.

Contribution

The paper proposes SHL0, a new $L_0$ penalty-based method with theoretical guarantees for interaction selection under hierarchical constraints.

Findings

SHL0 is consistent and has oracle properties.

SHL0 outperforms competitors in simulations.

The method is extendable to various ultrahigh-dimensional models.

Abstract

Selecting interactions from an ultrahigh-dimensional statistical model with $n$ observations and $p$ variables when $p\gg n$ is difficult because the number of candidates for interactions is $p(p-1)/2$ and a selected model should satisfy the strong hierarchical (SH) restriction. A new method called the SHL0 is proposed to overcome the difficulty. The objective function of the SHL0 method is composed of a loglikelihood function and an $L_0$ penalty. A well-known approach in theoretical computer science called local combinatorial optimization is used to optimize the objective function. We show that any local solution of the SHL0 is consistent and enjoys the oracle properties, implying that it is unnecessary to use a global solution in practice. Three additional advantages are: a tuning parameter is used to penalize the main effects and interactions; a closed-form expression can derive the tuning parameter; and the idea can be extended to arbitrary ultrahigh-dimensional statistical models. The proposed method is more flexible than the previous methods for selecting interactions. A simulation study of the research shows that the proposed SHL0 outperforms its competitors.