
TL;DR
This paper introduces a novel iterative method inspired by the Kaczmarz algorithm, involving projections between random hyperplanes, which accelerates the growth of small singular values while preserving the original solution.
Contribution
It proposes a new hyperplane projection technique that modifies the linear system to enhance convergence properties and spectral characteristics.
Findings
Small singular values grow exponentially with iterations
Method preserves the original solution
Accelerates convergence of linear system solutions
Abstract
The Kaczmarz method is a way to iteratively solve a linear system of equations . One interprets the solution as the point where hyperplanes intersect and then iteratively projects an approximate solution onto these hyperplanes to get better and better approximations. We note a somewhat related idea: one could take two random hyperplanes and project one into the orthogonal complement of the other. This leads to a sequence of linear systems which is fast to compute, preserves the original solution and whose small singular values grow like .
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic Gradient Optimization Techniques · Gaussian Processes and Bayesian Inference · Matrix Theory and Algorithms
