Greedy Block Gauss-Seidel Methods for Solving Large Linear Least Squares Problem

TL;DR

This paper introduces a greedy block Gauss-Seidel method for large linear least squares problems, demonstrating faster convergence and efficiency improvements over existing methods, including a pseudoinverse-free variant suitable for distributed computing.

Contribution

The paper proposes a novel greedy block Gauss-Seidel method with improved convergence and a pseudoinverse-free version for enhanced efficiency and distributed implementation.

Findings

The GBGS method has a smaller convergence factor than the greedy randomized coordinate descent.

The pseudoinverse-free GBGS method avoids computing Moore-Penrose pseudoinverses, accelerating computation.

Numerical experiments show our methods outperform GRCD in iteration count and computational time.

Abstract

With a greedy strategy to construct control index set of coordinates firstly and then choosing the corresponding column submatrix in each iteration, we present a greedy block Gauss-Seidel (GBGS) method for solving large linear least squares problem. Theoretical analysis demonstrates that the convergence factor of the GBGS method can be much smaller than that of the greedy randomized coordinate descent (GRCD) method proposed recently in the literature. On the basis of the GBGS method, we further present a pseudoinverse-free greedy block Gauss-Seidel method, which doesn't need to calculate the Moore-Penrose pseudoinverse of the column submatrix in each iteration any more and hence can be achieved greater acceleration. Moreover, this method can also be used for distributed implementations. Numerical experiments show that, for the same accuracy, our methods can far outperform the GRCD method in terms of the iteration number and computing time.