Inference for $L_2$-Boosting

TL;DR

This paper develops a statistical inference framework for $L_2$-Boosting, enabling valid hypothesis testing and confidence intervals for model components in high-dimensional additive models, with applications to real estate data.

Contribution

It introduces new inference methods tailored for the iterative, variable-selection process of $L_2$-Boosting, extending post-selection inference to this context.

Findings

Valid tests and confidence intervals for $L_2$-Boosting components.

Framework applicable to slow-learning algorithms and complex response sets.

Empirical validation on residential apartment sales price data.

Abstract

We propose a statistical inference framework for the component-wise functional gradient descent algorithm (CFGD) under normality assumption for model errors, also known as $L_2$-Boosting. The CFGD is one of the most versatile tools to analyze data, because it scales well to high-dimensional data sets, allows for a very flexible definition of additive regression models and incorporates inbuilt variable selection. Due to the variable selection, we build on recent proposals for post-selection inference. However, the iterative nature of component-wise boosting, which can repeatedly select the same component to update, necessitates adaptations and extensions to existing approaches. We propose tests and confidence intervals for linear, grouped and penalized additive model components selected by $L_2$-Boosting. Our concepts also transfer to slow-learning algorithms more generally, and to other selection techniques which restrict the response space to more complex sets than polyhedra. We apply our framework to an additive model for sales prices of residential apartments and investigate the properties of our concepts in simulation studies.