Robust linear regression with broad distributions of errors

TL;DR

This paper develops robust linear regression methods capable of handling broad, heavy-tailed noise distributions, such as alpha-stable, where traditional Gaussian-based methods fail, especially with outliers and small samples.

Contribution

It introduces approaches that minimize the width of residual distributions, enabling effective regression with non-Gaussian, heavy-tailed noise, and demonstrates their effectiveness through numerical examples.

Findings

Robust regression methods work with alpha-stable noise distributions.

Methods are effective with small samples and large outliers.

Equivalent to least squares under Gaussian noise.

Abstract

We consider the problem of linear fitting of noisy data in the case of broad (say $\alpha$-stable) distributions of random impacts ("noise"), which can lack even the first moment. This situation, common in statistical physics of small systems, in Earth sciences, in network science or in econophysics, does not allow for application of conventional Gaussian maximum-likelihood estimators resulting in usual least-squares fits. Such fits lead to large deviations of fitted parameters from their true values due to the presence of outliers. The approaches discussed here aim onto the minimization of the width of the distribution of residua. The corresponding width of the distribution can either be defined via the interquantile distance of the corresponding distributions or via the scale parameter in its characteristic function. The methods provide the robust regression even in the case of short samples with large outliers, and are equivalent to the normal least squares fit for the Gaussian noises. Our discussion is illustrated by numerical examples.