Strong Oracle Guarantees for Partial Penalized Tests of High Dimensional Generalized Linear Models

TL;DR

This paper develops a new algorithmic approach for high-dimensional generalized linear models that ensures theoretical guarantees for partial penalized tests, aligning practical computation with statistical theory.

Contribution

It introduces local linear approximation algorithms that converge to oracle estimators, bridging the gap between computation and theory for partial penalized tests.

Findings

LLA algorithms converge to oracle estimators with high probability

Test statistics are approximately chi-square in large samples

Simulation studies show LLA-based tests match oracle-based tests

Abstract

Partial penalized tests provide flexible approaches to testing linear hypotheses in high dimensional generalized linear models. However, because the estimators used in these tests are local minimizers of potentially non-convex folded-concave penalized objectives, the solutions one computes in practice may not coincide with the unknown local minima for which we have nice theoretical guarantees. To close this gap between theory and computation, we introduce local linear approximation (LLA) algorithms to compute the full and reduced model estimators for these tests and develop theory specifically for the LLA solutions. We prove that our LLA algorithms converge to oracle estimators for the full and reduced models in two steps with overwhelming probability. We then leverage this strong oracle result and the asymptotic properties of the oracle estimators to show that the partial penalized test statistics evaluated at the two-step LLA solutions are approximately chi-square in large samples, giving us guarantees for the tests using specific computed solutions and thereby closing the theoretical gap. We conduct simulation studies to assess the finite-sample performance of our testing procedures, finding that partial penalized tests using the LLA solutions agree with tests using the oracle estimators, and demonstrate our testing procedures in a real data application.