Robust Hierarchical-Optimization RLS Against Sparse Outliers

TL;DR

This paper introduces a robust hierarchical-optimization recursive least squares method that effectively estimates linear models contaminated with sparse outliers, improving robustness and convergence guarantees without heavy computational costs.

Contribution

It develops a novel outlier-robust HO-RLS algorithm that handles colored noise, avoids matrix inversion, and provides theoretical convergence guarantees in a probabilistic framework.

Findings

Significant performance improvements over existing methods in synthetic data tests.

Effective handling of sparse outliers in both stationary and non-stationary scenarios.

Low computational complexity due to gradient-based updates without matrix inversion.

Abstract

This paper fortifies the recently introduced hierarchical-optimization recursive least squares (HO-RLS) against outliers which contaminate infrequently linear-regression models. Outliers are modeled as nuisance variables and are estimated together with the linear filter/system variables via a sparsity-inducing (non-)convexly regularized least-squares task. The proposed outlier-robust HO-RLS builds on steepest-descent directions with a constant step size (learning rate), needs no matrix inversion (lemma), accommodates colored nominal noise of known correlation matrix, exhibits small computational footprint, and offers theoretical guarantees, in a probabilistic sense, for the convergence of the system estimates to the solutions of a hierarchical-optimization problem: Minimize a convex loss, which models a-priori knowledge about the unknown system, over the minimizers of the classical ensemble LS loss. Extensive numerical tests on synthetically generated data in both stationary and non-stationary scenarios showcase notable improvements of the proposed scheme over state-of-the-art techniques.