A New Combination of Preconditioned Gradient Descent Methods and Vector Extrapolation Techniques for Nonlinear Least-Squares Problems

TL;DR

This paper introduces a hybrid framework combining preconditioned gradient descent and vector extrapolation techniques to enhance convergence speed and solution accuracy in nonlinear least-squares problems, supported by extensive numerical experiments.

Contribution

It proposes a novel combination of extrapolation methods with preconditioned gradient descent, improving both convergence and approximation accuracy for nonlinear least-squares problems.

Findings

Extrapolation techniques improve convergence rate.

Hybrid methods reduce iteration counts and computational times.

Enhanced solution accuracy demonstrated in numerical experiments.

Abstract

Vector extrapolation methods are widely used in large-scale simulation studies, and numerous extrapolation-based acceleration techniques have been developed to enhance the convergence of linear and nonlinear fixed-point iterative methods. While classical extrapolation strategies often reduce the number of iterations or the computational cost, they do not necessarily lead to a significant improvement in the accuracy of the computed approximations. In this paper, we study the combination of preconditioned gradient-based methods with extrapolation strategies and propose an extrapolation-accelerated framework that simultaneously improves convergence and approximation accuracy. The focus is on the solution of nonlinear least-squares problems through the integration of vector extrapolation techniques with preconditioned gradient descent methods. A comprehensive set of numerical experiments is carried out to study the behavior of polynomial-type extrapolation methods and the vector $\varepsilon$-algorithm when coupled with gradient descent schemes, with and without preconditioning. The results demonstrate the impact of extrapolation techniques on both convergence rate and solution accuracy, and report iteration counts, computational times, and relative reconstruction errors. The performance of the proposed hybrid approaches is further assessed through a benchmarking study against Gauss--Newton methods based on generalized Krylov subspaces.