Sparse Estimation of Generalized Linear Models (GLM) via Approximated Information Criteria

TL;DR

This paper introduces MIC, a novel sparse estimation method for GLMs that uses an approximation of the information criterion, enabling efficient, tuning-free, and statistically valid sparse modeling.

Contribution

The paper presents MIC, a new method that approximates the information criterion for sparse GLM estimation, with a reparameterization for valid inference and no need for tuning parameters.

Findings

MIC outperforms existing methods in sparse estimation accuracy.

MIC is computationally efficient and tuning-free.

MIC provides valid significance testing results post-selection.

Abstract

We propose a new sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the $\ell_0$-norm with a continuous unit dent function. Besides, a reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results that are free of post-selection inference. We explore the asymptotic properties of MIC and illustrate its usage with both simulated experiments and empirical examples.