Greedy double subspaces coordinate descent method via orthogonalization

TL;DR

This paper introduces a novel coordinate descent method that iteratively projects onto solution spaces formed by two selected columns using Gram-Schmidt orthogonalization, improving solutions for highly coherent matrices.

Contribution

It proposes a simple block coordinate descent approach involving two active columns and provides convergence analysis and numerical validation for highly coherent matrices.

Findings

Effective for matrices with highly coherent columns

Converges under specified conditions

Numerical simulations confirm improved performance

Abstract

The coordinate descent method is an effective iterative method for solving large linear least-squares problems. In this paper, for the highly coherent columns case, we construct an effective coordinate descent method which iteratively projects the estimate onto a solution space formed by two greedily selected hyperplanes via Gram-Schmidt orthogonalization. Our methods may be regarded as a simple block version of coordinate descent method which involves two active columns. The convergence analysis of this method is provided and numerical simulations also confirm the effectiveness for matrices with highly coherent columns.