Variance function estimation in regression model via aggregation procedures

TL;DR

This paper introduces aggregation-based methods for estimating the variance function in regression models, demonstrating their consistency and effectiveness in heteroscedastic settings and regression with reject options.

Contribution

It proposes a two-step aggregation approach using independent samples for variance function estimation, with proven consistency for model selection and convex aggregation.

Findings

Methods are consistent in L2-error for both MS and C aggregations.

Performance demonstrated in heteroscedastic regression models.

Applicable to regression with reject options.

Abstract

In the regression problem, we consider the problem of estimating the variance function by the means of aggregation methods. We focus on two particular aggregation setting: Model Selection aggregation (MS) and Convex aggregation (C) where the goal is to select the best candidate and to build the best convex combination of candidates respectively among a collection of candidates. In both cases, the construction of the estimator relies on a two-step procedure and requires two independent samples. The first step exploits the first sample to build the candidate estimators for the variance function by the residual-based method and then the second dataset is used to perform the aggregation step. We show the consistency of the proposed method with respect to the L 2error both for MS and C aggregations. We evaluate the performance of these two methods in the heteroscedastic model and illustrate their interest in the regression problem with reject option.