A Modified Proximal Bundle Method Under A Frank-Wolfe Perspective

TL;DR

This paper introduces a modified proximal bundle method, MPB-FA, with fixed accuracy and parameters, linking it to Frank-Wolfe algorithms, and provides improved convergence complexity analysis for nonsmooth convex optimization.

Contribution

It proposes MPB-FA, a variant of PBM with fixed accuracy and parameters, and establishes a novel $ ilde{O}( ext{epsilon}^{-4/5})$ complexity bound, improving previous results.

Findings

MPB-FA's null steps can be viewed as a Frank-Wolfe algorithm.

Extended linear convergence of Kelley’s method to general convex piecewise linear functions.

Derived an improved $ ilde{O}( ext{epsilon}^{-4/5})$ iteration complexity bound.

Abstract

The proximal bundle method (PBM) is a fundamental and computationally effective algorithm for solving optimization problems with nonsmooth components. In this paper, we conduct a theoretical investigation of a modified proximal bundle method, which we call the Modified Proximal Bundle with Fixed Absolute Accuracy (MPB-FA). MPB-FA modifies PBM in two key aspects. Firstly, the null-step test of MPB-FA is based on an absolute accuracy criterion, and the accuracy is fixed over iterations, while the standard PBM uses a relative accuracy in the null-step test, which changes with iterations. Secondly, the proximal parameter in MPB-FA is also fixed over iterations, while it is permitted to change in the standard PBM. These modifications allow us to interpret a sequence of null steps of MPB-FA as a Frank-Wolfe algorithm on the Moreau envelope of the dual problem. In light of this correspondence, we first extend the linear convergence of Kelley's method on convex piecewise linear functions from the positive homogeneous to the general case. Building on this result, we propose a novel complexity analysis of MPB-FA and derive an $\mathcal{O}(\epsilon^{-4/5})$ iteration complexity, improving upon the best known $\mathcal{O}(\epsilon^{-2})$ guarantee on a related variant of PBM. It is worth-noting that the best known complexity bound for the classic PBM is $\mathcal{O}(\epsilon^{-3})$. Our approach also reveals new insights on bundle management.