Enhanced randomized Douglas-Rachford method: Improved probabilities and adaptive momentum

TL;DR

This paper introduces enhancements to the randomized Douglas-Rachford method by incorporating advanced sampling strategies and adaptive momentum, leading to improved convergence guarantees and practical performance in large-scale linear system solutions.

Contribution

The paper presents novel sampling techniques and an adaptive momentum scheme for RDR, achieving stronger convergence guarantees and better empirical performance.

Findings

Enhanced RDR with volume and without-replacement sampling improves convergence.

Adaptive momentum scheme accelerates convergence and adapts to problem dynamics.

Numerical results show the method outperforms original RDR across various problems.

Abstract

Randomized iterative methods have gained recent interest in machine learning and signal processing for solving large-scale linear systems. One such example is the randomized Douglas-Rachford (RDR) method, which updates the iterate by reflecting it through two randomly selected hyperplanes and taking a convex combination with the current point. In this work, we enhance RDR by introducing improved sampling strategies and an adaptive heavy-ball momentum scheme. Specifically, we incorporate without-replacement and volume sampling into RDR, and establish stronger convergence guarantees compared to conventional i.i.d. sampling. Furthermore, we develop an adaptive momentum mechanism that dynamically adjusts step sizes and momentum parameters based on previous iterates, and prove that the resulting method achieves linear convergence in expectation with improved convergence bounds. Numerical experiments demonstrate that the enhanced RDR method consistently outperforms the original version, providing substantial practical benefits across a range of problem settings.