Primal-dual proximal bundle and conditional gradient methods for convex problems

TL;DR

This paper introduces primal-dual proximal bundle methods for convex nonsmooth optimization, establishing iteration-complexity and revealing duality with conditional gradient methods, supported by numerical experiments showing improved performance.

Contribution

It develops new primal-dual proximal bundle algorithms, analyzes their iteration complexity, and uncovers a duality between conditional gradient and cutting-plane schemes.

Findings

Proposed methods outperform subgradient methods in matrix game experiments.

Established iteration-complexity bounds for the new algorithms.

Discovered a duality linking conditional gradient and proximal bundle methods.

Abstract

This paper studies the primal-dual convergence and iteration-complexity of proximal bundle methods for solving nonsmooth problems with convex structures. More specifically, we develop a family of primal-dual proximal bundle methods for solving convex nonsmooth composite optimization problems and establish the iteration-complexity in terms of a primal-dual gap. We also propose a class of proximal bundle methods for solving convex-concave nonsmooth composite saddle-point problems and establish the iteration-complexity to find an approximate saddle-point. This paper places special emphasis on the primal-dual perspective of the proximal bundle method. In particular, we discover an interesting duality between the conditional gradient method and the cutting-plane scheme used within the proximal bundle method. Leveraging this duality, we further develop novel variants of both the conditional gradient method and the cutting-plane scheme. Additionally, we report numerical experiments to demonstrate the effectiveness and efficiency of the proposed proximal bundle methods in comparison with the subgradient method for solving a regularized matrix game.