Randomized block coordinate descent method for linear ill-posed problems

TL;DR

This paper analyzes the convergence of a randomized block coordinate descent method for large-scale linear ill-posed problems, demonstrating weak and strong convergence under various stopping rules and regularization, with applications in imaging.

Contribution

It provides the first convergence analysis of RBCD for ill-posed problems, including noisy data handling, stopping rules, and regularization techniques.

Findings

Weak convergence to solutions under a priori stopping rules

Finite termination using discrepancy principle with a posteriori rule

Strong convergence results for tensor product structured problems

Abstract

Consider the linear ill-posed problems of the form $\sum_{i=1}^{b} A_i x_i =y$, where, for each $i$, $A_i$ is a bounded linear operator between two Hilbert spaces $X_i$ and ${\mathcal Y}$. When $b$ is huge, solving the problem by an iterative method using the full gradient at each iteration step is both time-consuming and memory insufficient. Although randomized block coordinate decent (RBCD) method has been shown to be an efficient method for well-posed large-scale optimization problems with a small amount of memory, there still lacks a convergence analysis on the RBCD method for solving ill-posed problems. In this paper, we investigate the convergence property of the RBCD method with noisy data under either {\it a priori} or {\it a posteriori} stopping rules. We prove that the RBCD method combined with an {\it a priori} stopping rule yields a sequence that converges weakly to a solution of the problem almost surely. We also consider the early stopping of the RBCD method and demonstrate that the discrepancy principle can terminate the iteration after finite many steps almost surely. For a class of ill-posed problems with special tensor product form, we obtain strong convergence results on the RBCD method. Furthermore, we consider incorporating the convex regularization terms into the RBCD method to enhance the detection of solution features. To illustrate the theory and the performance of the method, numerical simulations from the imaging modalities in computed tomography and compressive temporal imaging are reported.