Recursive Bias Estimation and $L_2$ Boosting

TL;DR

This paper introduces a bias correction method for regression smoothers that aligns with the $L_2$ Boosting algorithm, offering new insights into its behavior and practical stopping rules through theoretical analysis and simulations.

Contribution

It provides a new statistical interpretation of $L_2$ Boosting as an iterative bias correction process and analyzes its convergence properties for common smoothers.

Findings

Boosting can produce divergent sequences depending on the smoother.

A stopping rule is recommended to prevent divergence.

Simulation results demonstrate the effectiveness of the iterative smoother.

Abstract

This paper presents a general iterative bias correction procedure for regression smoothers. This bias reduction schema is shown to correspond operationally to the $L_2$ Boosting algorithm and provides a new statistical interpretation for $L_2$ Boosting. We analyze the behavior of the Boosting algorithm applied to common smoothers $S$ which we show depend on the spectrum of $I-S$. We present examples of common smoother for which Boosting generates a divergent sequence. The statistical interpretation suggest combining algorithm with an appropriate stopping rule for the iterative procedure. Finally we illustrate the practical finite sample performances of the iterative smoother via a simulation study. simulations.