On Motzkin's Method for Inconsistent Linear Systems

TL;DR

This paper analyzes Motzkin's method for solving large-scale inconsistent linear systems, showing it accelerates convergence initially based on residual dynamics, with experimental validation on synthetic and real data.

Contribution

The paper provides a theoretical analysis of Motzkin's method's initial acceleration and quantifies its effects for Gaussian systems, supported by experimental evidence.

Findings

Motzkin's method accelerates convergence initially.

Acceleration depends on the residual's dynamic range.

Experimental results support the theoretical analysis.

Abstract

Iterative linear solvers have gained recent popularity due to their computational efficiency and low memory footprint for large-scale linear systems. The relaxation method, or Motzkin's method, can be viewed as an iterative method that projects the current estimation onto the solution hyperplane corresponding to the most violated constraint. Although this leads to an optimal selection strategy for consistent systems, for inconsistent least square problems, the strategy presents a tradeoff between convergence rate and solution accuracy. We provide a theoretical analysis that shows Motzkin's method offers an initially accelerated convergence rate and this acceleration depends on the dynamic range of the residual. We quantify this acceleration for Gaussian systems as a concrete example. Lastly, we include experimental evidence on real and synthetic systems that support the analysis.