Nesterov's Accelerated Jacobi-Type Methods for Large-scale Symmetric Positive Semidefinite Linear Systems

TL;DR

This paper introduces an accelerated Jacobi-type method using Nesterov's technique for large-scale symmetric positive semidefinite systems, achieving faster convergence and better parallel scalability than classical methods.

Contribution

It proposes a novel Nesterov-accelerated Jacobi method with proven convergence rates and a restarted variant, enhancing efficiency and parallel performance for large-scale problems.

Findings

Achieves $O(t^{-2})$ convergence rate in objective function values.

Outperforms classical Jacobi and conjugate gradient methods in experiments.

Demonstrates high scalability with parallel implementation.

Abstract

Solving symmetric positive semidefinite linear systems is an essential task in many scientific computing problems. While Jacobi-type methods, including the classical Jacobi method and the weighted Jacobi method, exhibit simplicity in their forms and friendliness to parallelization, they are not attractive either because of the potential convergence failure or their slow convergence rate. This paper aims to showcase the possibility of improving classical Jacobi-type methods by employing Nesterov's acceleration technique that results in an accelerated Jacobi-type method with improved convergence properties. Simultaneously, it preserves the appealing features for parallel implementation. In particular, we show that the proposed method has an $O\left(t^{-2}\right)$ convergence rate in terms of objective function values of the associated convex quadratic optimization problem, where $t\geq 1$ denotes the iteration counter. To further improve the practical performance of the proposed method, we also develop and analyze a restarted variant of the method, which is shown to have an $O\left((\log_2(t))^2t^{-2}\right)$ convergence rate when the coefficient matrix is positive definite. Furthermore, we conduct appropriate numerical experiments to evaluate the efficiency of the proposed method. Our numerical results demonstrate that the proposed method outperforms the classical Jacobi-type methods and the conjugate gradient method, and shows a comparable performance as the preconditioned conjugate gradient method with a diagonal preconditioner. Finally, we develop a parallel implementation and conduct speed-up tests on some large-scale systems. Our results indicate that the proposed framework is highly scalable.