Optimal Convergence Rates for the Proximal Bundle Method

TL;DR

This paper analyzes the convergence rates of the proximal bundle method for nonsmooth convex optimization, proposing adaptive and parameter-free schemes with near-optimal rates and practical parallelizable variants.

Contribution

It introduces nonconstant stepsize schemes and a parallelizable variant that adapt to problem smoothness and do not require prior knowledge of function parameters.

Findings

Adaptive convergence rates for the bundle method in various settings.

Proposed nonconstant stepsize schemes achieve optimal convergence.

Parallelizable variant maintains near-optimal rates with minimal practical overhead.

Abstract

We study convergence rates of the classic proximal bundle method for a variety of nonsmooth convex optimization problems. We show that, without any modification, this algorithm adapts to converge faster in the presence of smoothness or a H\"older growth condition. Our analysis reveals that with a constant stepsize, the bundle method is adaptive, yet it exhibits suboptimal convergence rates. We overcome this shortcoming by proposing nonconstant stepsize schemes with optimal rates. These schemes use function information such as growth constants, which might be prohibitive in practice. We provide a parallelizable variant of the bundle method that can be applied without prior knowledge of function parameters while maintaining near-optimal rates. The practical impact of this scheme is limited since we incur a (parallelizable) log factor in the complexity. These results improve on the scarce existing convergence rates and provide a unified analysis approach across problem settings and algorithmic details. Numerical experiments support our findings.