A survey of scalar and vector extrapolation

TL;DR

This paper provides a comprehensive review of scalar and vector extrapolation methods, detailing their theoretical foundations, historical development, and applications in modern computational algorithms to improve convergence and efficiency.

Contribution

It offers a unified overview of classical and modern extrapolation techniques, connecting historical methods with current computational practices and applications.

Findings

Detailed analysis of convergence properties and stability

Coverage of modern applications in iterative solvers and simulations

Comparison of scalar and vector extrapolation methods

Abstract

Scalar extrapolation and convergence acceleration methods are central tools in numerical analysis for improving the efficiency of iterative algorithms and the summation of slowly convergent series. These methods construct transformed sequences that converge more rapidly to the same limit without altering the underlying iterative process, thereby reducing computational cost and enhancing numerical accuracy. Historically, the origins of such techniques can be traced back to classical algebraic methods by AlKhwarizmi and early series acceleration techniques by Newton, while systematic approaches emerged in the 20th century with Aitken process and Richardson extrapolation. Later developments, including the Shanks transformation and Wynn epsilon algorithm, provided general frameworks capable of eliminating multiple dominant error components, with deep connections to Pade approximants and rational approximations of generating functions. This paper presents a comprehensive review of classical scalar extrapolation methods, including Richardson extrapolation, Aitken process, Shanks transformation, Wynn epsilon algorithm, and other algorithms. We examine their theoretical foundations, asymptotic error models, convergence properties, numerical stability, and practical implementation considerations. The second part of this work is dedicated to vector extrapolation methods: polynomial based ones and epsilon algorithm generalizations to vector sequences. Additionally, we highlight modern developments such as their applications to iterative solvers, Krylov subspace methods, and large-scale computational simulations. The aim of this review is to provide a unified perspective on scalar and vector extrapolation techniques, bridging historical origins, theoretical insights, and contemporary computational applications.