Implementation of an Oracle-Structured Bundle Method for Distributed Optimization

TL;DR

This paper introduces a new bundle method tailored for distributed convex optimization problems with costly oracle evaluations, combining multiple techniques to improve reliability, parameter simplicity, and convergence speed.

Contribution

It develops an oracle-structured bundle method that integrates known techniques, offering practical advantages over existing methods for distributed convex optimization.

Findings

Reliable across various applications

Requires minimal parameter tuning

Achieves approximate solutions in few tens of iterations

Abstract

We consider the problem of minimizing a function that is a sum of convex agent functions plus a convex common public function that couples them. The agent functions can only be accessed via a subgradient oracle; the public function is assumed to be structured and expressible in a domain specific language (DSL) for convex optimization. We focus on the case when the evaluation of the agent oracles can require significant effort, which justifies the use of solution methods that carry out significant computation in each iteration. To solve this problem we integrate multiple known techniques (or adaptations of known techniques) for bundle-type algorithms, obtaining a method which has a number of practical advantages over other methods that are compatible with our access methods, such as proximal subgradient methods. First, it is reliable, and works well across a number of applications. Second, it has very few parameters that need to be tuned, and works well with sensible default values. Third, it typically produces a reasonable approximate solution in just a few tens of iterations. This paper is accompanied by an open-source implementation of the proposed solver, available at \url{https://github.com/cvxgrp/OSBDO}.