Bilevel Distributed Optimization in Directed Networks

TL;DR

This paper introduces a novel bilevel distributed optimization framework for directed networks, employing a regularization-based approach and gradient tracking to achieve convergence in complex constrained and unconstrained problems.

Contribution

It develops a new unifying bilevel optimization model for directed networks and proposes an iteratively regularized push-pull gradient algorithm with convergence guarantees.

Findings

Established convergence rates for suboptimality and infeasibility.

Proposed algorithm outperforms existing methods in numerical tests.

Unified approach applies to multiple classes of distributed optimization problems.

Abstract

Motivated by emerging applications in wireless sensor networks and large-scale data processing, we consider distributed optimization over directed networks where the agents communicate their information locally to their neighbors to cooperatively minimize a global cost function. We introduce a new unifying distributed constrained optimization model that is characterized as a bilevel optimization problem. This model captures a wide range of existing problems over directed networks including: (i) Distributed optimization with linear constraints; (ii) Distributed unconstrained nonstrongly convex optimization over directed networks. Employing a novel regularization-based relaxation approach and gradient-tracking schemes, we develop an iteratively regularized push-pull gradient algorithm. We establish the consensus and derive new convergence rate statements for suboptimality and infeasibility of the generated iterates for solving the bilevel model. The proposed algorithm and the complexity analysis obtained in this work appear to be new for addressing the bilevel model and also for the two sub-classes of problems. The numerical performance of the proposed algorithm is presented.