Fast and stable modification of the Gauss-Newton method for low-rank signal estimation

TL;DR

This paper introduces a modified Gauss-Newton method for low-rank signal estimation that is both fast and numerically stable, capable of handling long time series and missing data efficiently.

Contribution

A novel modification of the weighted Gauss-Newton method that enables direct low-rank projection, improving stability and speed for long time series with autoregressive noise.

Findings

Computational cost scales as O(N r^2 + N p^2 + r N log N) for large N.

Method handles missing data without additional computational burden.

Compared to state-of-the-art, it offers enhanced numerical stability and efficiency.

Abstract

The weighted nonlinear least-squares problem for low-rank signal estimation is considered. The problem of constructing a numerical solution that is stable and fast for long time series is addressed. A modified weighted Gauss-Newton method, which can be implemented through the direct variable projection onto a space of low-rank signals, is proposed. For a weight matrix which provides the maximum likelihood estimator of the signal in the presence of autoregressive noise of order $p$ the computational cost of iterations is $O(N r^2 + N p^2 + r N \log N)$ as $N$ tends to infinity, where $N$ is the time-series length, $r$ is the rank of the approximating time series. Moreover, the proposed method can be applied to data with missing values, without increasing the computational cost. The method is compared with state-of-the-art methods based on the variable projection approach in terms of floating-point numerical stability and computational cost.