Mean and variance estimation in high-dimensional heteroscedastic models with non-convex penalties

TL;DR

This paper addresses high-dimensional heteroscedastic regression by proposing a two-step estimation method using non-convex penalties, achieving oracle properties for variance and mean estimates.

Contribution

It introduces a novel post-Lasso approach for variance estimation in heteroscedastic models with theoretical guarantees and empirical validation.

Findings

Oracle properties for variance and mean estimators

Effective handling of heteroscedasticity in high dimensions

Empirical results support theoretical claims

Abstract

Despite its prevalence in statistical datasets, heteroscedasticity (non-constant sample variances) has been largely ignored in the high-dimensional statistics literature. Recently, studies have shown that the Lasso can accommodate heteroscedastic errors, with minor algorithmic modifications (Belloni et al., 2012; Gautier and Tsybakov, 2013). In this work, we study heteroscedastic regression with linear mean model and log-linear variances model with sparse high-dimensional parameters. In this work, we propose estimating variances in a post-Lasso fashion, which is followed by weighted-least squares mean estimation. These steps employ non-convex penalties as in Fan and Li (2001), which allows us to prove oracle properties for both post-Lasso variance and mean parameter estimates. We reinforce our theoretical findings with experiments.