Least Square Error Method Robustness of Computation: What is not usually considered and taught

TL;DR

This paper discusses the numerical instability issues in Least Square Error (LSE) methods with large data spans and proposes a simple, orthogonal basis vector approach to improve stability in linear regression and radial basis function approximation.

Contribution

It introduces a novel LSE computation method using orthogonal basis vectors to enhance numerical stability for large span data sets.

Findings

Improved stability in LSE computations with large data spans.

Effective for linear regression and radial basis function approximation.

Reduces issues with ill-conditioned pseudoinverse matrices.

Abstract

There are many practical applications based on the Least Square Error (LSE) approximation. It is based on a square error minimization 'on a vertical' axis. The LSE method is simple and easy also for analytical purposes. However, if data span is large over several magnitudes or non-linear LSE is used, severe numerical instability can be expected. The presented contribution describes a simple method for large span of data LSE computation. It is especially convenient if large span of data are to be processed, when the 'standard' pseudoinverse matrix is ill conditioned. It is actually based on a LSE solution using orthogonal basis vectors instead of orthonormal basis vectors. The presented approach has been used for a linear regression as well as for approximation using radial basis functions.