Robust Linear Regression Analysis - A Greedy Approach

TL;DR

This paper introduces GARD, a greedy algorithm for robust linear regression that effectively identifies outliers and recovers signals, with proven theoretical guarantees and demonstrated superior performance through simulations.

Contribution

A novel greedy algorithm (GARD) for robust linear regression that explicitly models outliers using sparsity and provides theoretical recovery guarantees.

Findings

GARD accurately identifies outliers and recovers signals in noisy data.

Theoretical bounds guarantee exact recovery under certain conditions.

Simulations show GARD outperforms existing methods in robustness and efficiency.

Abstract

The task of robust linear estimation in the presence of outliers is of particular importance in signal processing, statistics and machine learning. Although the problem has been stated a few decades ago and solved using classical (considered nowadays) methods, recently it has attracted more attention in the context of sparse modeling, where several notable contributions have been made. In the present manuscript, a new approach is considered in the framework of greedy algorithms. The noise is split into two components: a) the inlier bounded noise and b) the outliers, which are explicitly modeled by employing sparsity arguments. Based on this scheme, a novel efficient algorithm (Greedy Algorithm for Robust Denoising - GARD), is derived. GARD alternates between a least square optimization criterion and an Orthogonal Matching Pursuit (OMP) selection step that identifies the outliers. The case where only outliers are present has been studied separately, where bounds on the \textit{Restricted Isometry Property} guarantee that the recovery of the signal via GARD is exact. Moreover, theoretical results concerning convergence as well as the derivation of error bounds in the case of additional bounded noise are discussed. Finally, we provide extensive simulations, which demonstrate the comparative advantages of the new technique.