Fast and Exact Least Absolute Deviations Line Fitting via Piecewise Affine Lower-Bounding

TL;DR

The paper introduces PALB, an exact, efficient algorithm for LAD line fitting that outperforms existing LP and IRLS methods, with proven correctness and practical implementation.

Contribution

PALB is a novel piecewise affine lower-bounding method that provides the first exact, fast algorithm for LAD line fitting with publicly available implementation.

Findings

PALB exhibits empirical log-linear scaling on synthetic and real datasets.

PALB is consistently faster than existing LP and IRLS solvers.

The method guarantees exact solutions for LAD line fitting.

Abstract

Least-absolute-deviations (LAD) line fitting is robust to outliers but computationally more involved than least squares regression. Although the literature includes linear and near-linear time algorithms for the LAD line fitting problem, these methods are difficult to implement and, to our knowledge, lack maintained public implementations. As a result, practitioners often resort to linear programming (LP) based methods such as the simplex-based Barrodale-Roberts method and interior-point methods, or on iteratively reweighted least squares (IRLS) approximation which does not guarantee exact solutions. To close this gap, we propose the Piecewise Affine Lower-Bounding (PALB) method, an exact algorithm for LAD line fitting. PALB uses supporting lines derived from subgradients to build piecewise-affine lower bounds, and employs a subdivision scheme involving minima of these lower bounds. We prove correctness and provide bounds on the number of iterations. On synthetic datasets with varied signal types and noise including heavy-tailed outliers as well as a real dataset from the NOAA's Integrated Surface Database, PALB exhibits empirical log-linear scaling. It is consistently faster than publicly available implementations of LP based and IRLS based solvers. We provide a reference implementation written in Rust with a Python API.