The Inexact Cyclic Block Proximal Gradient Method and Properties of Inexact Proximal Maps

TL;DR

This paper introduces the Inexact Cyclic Block Proximal Gradient (I-CBPG) method, allowing inexact computations while maintaining convergence, and demonstrates its practical efficiency through numerical experiments and theoretical analysis.

Contribution

It extends the CBPG method to inexact computations, providing convergence guarantees and a unified framework for inexact proximal maps and gradients.

Findings

I-CBPG maintains convergence with inexact computations.

Numerical experiments show practical computational advantages.

The $ ext{δ}$-Second Prox Theorem links inexact proximal maps to $ ext{δ}$-subgradients.

Abstract

This paper expands the Cyclic Block Proximal Gradient method for block separable composite minimization by allowing for inexactly computed gradients and proximal maps. The resultant algorithm, the Inexact Cyclic Block Proximal Gradient (I-CBPG) method, shares the same convergence rate as its exactly computed analogue provided the allowable errors decrease sufficiently quickly or are pre-selected to be sufficiently small. We provide numerical experiments that showcase the practical computational advantage of I-CBPG for certain fixed tolerances of approximation error and for a dynamically decreasing error tolerance regime in particular. We establish a tight relationship between inexact proximal map evaluations and $\delta$-subgradients in our $\delta$-Second Prox Theorem. This theorem forms the foundation of our convergence analysis and enables us to show that inexact gradient computations and other notions of inexact proximal map computation can be subsumed within a single unifying framework.