Structural Change in Sparsity

TL;DR

This paper introduces a penalized M-estimator that detects and models structural changes in sparsity within high-dimensional data, allowing for heterogeneity across sub-populations without prior knowledge of change points.

Contribution

It develops a novel estimator capable of identifying and adapting to structural changes in sparsity, with proven asymptotic properties and super-consistent threshold estimation.

Findings

Estimator achieves oracle properties.

Super-consistent threshold parameter estimation.

Effective in quantile and logistic regression models.

Abstract

In the high-dimensional sparse modeling literature, it has been crucially assumed that the sparsity structure of the model is homogeneous over the entire population. That is, the identities of important regressors are invariant across the population and across the individuals in the collected sample. In practice, however, the sparsity structure may not always be invariant in the population, due to heterogeneity across different sub-populations. We consider a general, possibly non-smooth M-estimation framework, allowing a possible structural change regarding the identities of important regressors in the population. Our penalized M-estimator not only selects covariates but also discriminates between a model with homogeneous sparsity and a model with a structural change in sparsity. As a result, it is not necessary to know or pretest whether the structural change is present, or where it occurs. We derive asymptotic bounds on the estimation loss of the penalized M-estimators, and achieve the oracle properties. We also show that when there is a structural change, the estimator of the threshold parameter is super-consistent. If the signal is relatively strong, the rates of convergence can be further improved and asymptotic distributional properties of the estimators including the threshold estimator can be established using an adaptive penalization. The proposed methods are then applied to quantile regression and logistic regression models and are illustrated via Monte Carlo experiments.