Analysis of the Forward Search using some new results for martingales and empirical processes

TL;DR

This paper provides a theoretical analysis of the Forward Search algorithm in regression, demonstrating its convergence to Gaussian processes using advanced martingale and empirical process techniques.

Contribution

It introduces new martingale inequalities and empirical process theory to analyze the convergence properties of the Forward Search algorithm.

Findings

Sequences of estimators converge to Gaussian processes

New martingale inequality developed for iterative processes

Empirical process theory extended to weighted and marked processes

Abstract

The Forward Search is an iterative algorithm for avoiding outliers in a regression analysis suggested by Hadi and Simonoff (J. Amer. Statist. Assoc. 88 (1993) 1264-1272), see also Atkinson and Riani (Robust Diagnostic Regression Analysis (2000) Springer). The algorithm constructs subsets of "good" observations so that the size of the subsets increases as the algorithm progresses. It results in a sequence of regression estimators and forward residuals. Outliers are detected by monitoring the sequence of forward residuals. We show that the sequences of regression estimators and forward residuals converge to Gaussian processes. The proof involves a new iterated martingale inequality, a theory for a new class of weighted and marked empirical processes, the corresponding quantile process theory, and a fixed point argument to describe the iterative aspect of the procedure.