DIAMOND: Taming Sample and Communication Complexities in Decentralized Bilevel Optimization

TL;DR

This paper introduces DIAMOND, a decentralized bilevel optimization algorithm that reduces sample and communication complexities using a single-loop structure, momentum, and gradient tracking, suitable for edge networks.

Contribution

The paper presents DIAMOND, a novel decentralized bilevel optimization method with a single-loop design that avoids full gradient evaluations, lowering complexities and outperforming existing approaches.

Findings

Achieves $ ilde{O}( ext{epsilon}^{-3/2})$ sample and communication complexity.

Does not require full gradient evaluations, reducing computational cost.

Experimental results confirm theoretical complexity improvements.

Abstract

Decentralized bilevel optimization has received increasing attention recently due to its foundational role in many emerging multi-agent learning paradigms (e.g., multi-agent meta-learning and multi-agent reinforcement learning) over peer-to-peer edge networks. However, to work with the limited computation and communication capabilities of edge networks, a major challenge in developing decentralized bilevel optimization techniques is to lower sample and communication complexities. This motivates us to develop a new decentralized bilevel optimization called DIAMOND (decentralized single-timescale stochastic approximation with momentum and gradient-tracking). The contributions of this paper are as follows: i) our DIAMOND algorithm adopts a single-loop structure rather than following the natural double-loop structure of bilevel optimization, which offers low computation and implementation complexity; ii) compared to existing approaches, the DIAMOND algorithm does not require any full gradient evaluations, which further reduces both sample and computational complexities; iii) through a careful integration of momentum information and gradient tracking techniques, we show that the DIAMOND algorithm enjoys $\mathcal{O}(\epsilon^{-3/2})$ in sample and communication complexities for achieving an $\epsilon$-stationary solution, both of which are independent of the dataset sizes and significantly outperform existing works. Extensive experiments also verify our theoretical findings.