Variance function estimation in high-dimensions

TL;DR

This paper introduces a novel non-convex penalized pseudolikelihood estimator for high-dimensional heteroscedastic regression, achieving oracle properties and effectively modeling non-constant error variances.

Contribution

It proposes the HIPPO method for joint mean and variance estimation in high dimensions, with proven oracle properties and demonstrated effectiveness on simulated and real data.

Findings

HIPPO achieves oracle properties in high-dimensional heteroscedastic regression.

The method performs well in simulations and real data applications.

Rates of convergence match those of the true model.

Abstract

We consider the high-dimensional heteroscedastic regression model, where the mean and the log variance are modeled as a linear combination of input variables. Existing literature on high-dimensional linear regres- sion models has largely ignored non-constant error variances, even though they commonly occur in a variety of applications ranging from biostatis- tics to finance. In this paper we study a class of non-convex penalized pseudolikelihood estimators for both the mean and variance parameters. We show that the Heteroscedastic Iterative Penalized Pseudolikelihood Optimizer (HIPPO) achieves the oracle property, that is, we prove that the rates of convergence are the same as if the true model was known. We demonstrate numerical properties of the procedure on a simulation study and real world data.