On Linear Convergence of Distributed Stochastic Bilevel Optimization over Undirected Networks via Gradient Aggregation

TL;DR

This paper introduces a distributed stochastic gradient aggregation method for bilevel optimization over networks, proving linear convergence under weak assumptions and demonstrating its effectiveness through numerical experiments.

Contribution

It presents a novel distributed stochastic gradient scheme with proven linear convergence under global strong convexity, relaxing previous local convexity requirements.

Findings

Algorithm converges linearly under global strong convexity.

Effective in distributed sensor network and linear regression problems.

Extends theoretical guarantees for distributed bilevel optimization.

Abstract

Many large-scale constrained optimization problems can be formulated as bilevel distributed optimization tasks over undirected networks, where agents collaborate to minimize a global cost function while adhering to constraints, relying only on local communication and computation. In this work, we propose a distributed stochastic gradient aggregation scheme and establish its linear convergence under the weak assumption of global strong convexity, which relaxes the common requirement of local function convexity on the objective and constraint functions. Specifically, we prove that the algorithm converges at a linear rate when the global objective function (and not each local objective function) satisfies strong-convexity. Our results significantly extend existing theoretical guarantees for distributed bilevel optimization. Additionally, we demonstrate the effectiveness of our approach through numerical experiments on distributed sensor network problems and distributed linear regression with rank-deficient data.