Tractable Post-Selection Maximum Likelihood Inference for the Lasso

TL;DR

This paper introduces a novel stochastic ascent method to perform valid post-selection maximum likelihood inference for the lasso, addressing intractability issues and improving estimation accuracy after model selection.

Contribution

It proposes a new approach using noisy unbiased score estimates and stochastic ascent to enable valid post-selection inference for the lasso.

Findings

Estimates are consistent and asymptotically normal.

Confidence intervals achieve near-nominal coverage.

Point estimates outperform lasso in sparse models.

Abstract

Applying standard statistical methods after model selection may yield inefficient estimators and hypothesis tests that fail to achieve nominal type-I error rates. The main issue is the fact that the post-selection distribution of the data differs from the original distribution. In particular, the observed data is constrained to lie in a subset of the original sample space that is determined by the selected model. This often makes the post-selection likelihood of the observed data intractable and maximum likelihood inference difficult. In this work, we get around the intractable likelihood by generating noisy unbiased estimates of the post-selection score function and using them in a stochastic ascent algorithm that yields correct post-selection maximum likelihood estimates. We apply the proposed technique to the problem of estimating linear models selected by the lasso. In an asymptotic analysis the resulting estimates are shown to be consistent for the selected parameters and to have a limiting truncated normal distribution. Confidence intervals constructed based on the asymptotic distribution obtain close to nominal coverage rates in all simulation settings considered, and the point estimates are shown to be superior to the lasso estimates when the true model is sparse.