Distributed Derivative-Free Optimization Using Inexact ADMM and Trust-Region Methods

TL;DR

This paper introduces a scalable distributed derivative-free optimization method combining inexact ADMM and trust-region techniques, suitable for high-dimensional black-box problems with linear constraints, and demonstrates its theoretical convergence and practical efficiency.

Contribution

It develops a novel two-level ADMM framework with inexact subproblem solutions for nonconvex derivative-free optimization under linear constraints, enhancing scalability and efficiency.

Findings

The method converges to an approximate solution with theoretical guarantees.

It outperforms monolithic derivative-free approaches on high-dimensional benchmarks.

Effective in distributed learning applications with complex constraints.

Abstract

To reduce complexity and achieve scalable performance in high-dimensional black-box settings, we propose a distributed method for nonconvex derivative-free optimization of continuous variables with an additively separable objective, subject to linear equality constraints. The approach is built upon the alternating direction method of multipliers (ADMM) as the distributed optimization framework. To handle general, potentially complicating linear equality constraints beyond the standard ADMM formulation, we employ a two-level ADMM structure: an inner layer that performs sequential ADMM updates, and an outer layer that drives an introduced slack variable to zero via the method of multipliers. In addition, each subproblem is solved inexactly using a derivative-free trust-region solver, ensuring suboptimality within a decreasing, theoretically controlled error tolerance. This inexactness is critical for both computational efficiency and practical applicability in black-box settings, where exact solutions are impractical or overly expensive. We establish theoretical convergence of the proposed approach to an approximate solution, and demonstrate improved computational efficiency over monolithic derivative-free optimization approaches on challenging high-dimensional benchmarks, as well as effective performance on a distributed learning problem.