Additive Schwarz Methods for Convex Optimization with Backtracking

TL;DR

This paper introduces a backtracking strategy for additive Schwarz methods that adaptively adjusts step sizes, significantly improving convergence rates for convex optimization problems and can be combined with momentum acceleration.

Contribution

It proposes a novel, solver-independent backtracking scheme for additive Schwarz methods, enhancing convergence speed and enabling further acceleration with momentum techniques.

Findings

Proven improved convergence rates with the new backtracking strategy.

The combined method with momentum acceleration shows further performance gains.

Numerical experiments validate the theoretical improvements.

Abstract

This paper presents a novel backtracking strategy for additive Schwarz methods for general convex optimization problems as an acceleration scheme. The proposed backtracking strategy is independent of local solvers, so that it can be applied to any algorithms that can be represented in an abstract framework of additive Schwarz methods. Allowing for adaptive increasing and decreasing of the step size along the iterations, the convergence rate of an algorithm is greatly improved. Improved convergence rate of the algorithm is proven rigorously. In addition, combining the proposed backtracking strategy with a momentum acceleration technique, we propose a further accelerated additive Schwarz method. Numerical results for various convex optimization problems that support our theory are presented.