Gauss-Seidel Method with Oblique Direction

TL;DR

This paper introduces the Gauss-Seidel method with oblique direction (GSO) and its randomized version (RGSO) for efficiently computing least-squares solutions, demonstrating improved convergence and efficiency over existing methods.

Contribution

The paper proposes a novel GSO method and its randomized variant, with theoretical convergence proofs and superior performance compared to coordinate descent methods.

Findings

GSO converges to the least-squares solution.

RGSO is also proven to converge.

Both methods outperform CD and RCD in efficiency.

Abstract

In this paper, a Gauss-Seidel method with oblique direction (GSO) is proposed for finding the least-squares solution to a system of linear equations, where the coefficient matrix may be full rank or rank deficient and the system is overdetermined or underdetermined. Through this method, the number of iteration steps and running time can be reduced to a greater extent to find the least-squares solution, especially when the columns of matrix A are close to linear correlation. It is theoretically proved that GSO method converges to the least-squares solution. At the same time, a randomized version--randomized Gauss-Seidel method with oblique direction (RGSO) is established, and its convergence is proved. Theoretical proof and numerical results show that the GSO method and the RGSO method are more efficient than the coordinate descent (CD) method and the randomized coordinate descent (RCD) method.