Decentralized Online Convex Optimization with Unknown Feedback Delays

TL;DR

This paper introduces a decentralized online convex optimization algorithm that adaptively handles unknown, varying feedback delays among agents, improving regret bounds and performance in networked learning scenarios.

Contribution

It proposes a novel delay-adaptive algorithm with decentralized delay estimation, achieving tighter regret bounds without prior delay knowledge, applicable to both convex and strongly convex settings.

Findings

Achieves regret bound of O(N√d_tot + N√T(1-σ2)^{1/4})

Extends to strongly convex functions with sharper regret bounds

Experimental results show improved performance over benchmarks

Abstract

Decentralized online convex optimization (D-OCO), where multiple agents within a network collaboratively learn optimal decisions in real-time, arises naturally in applications such as federated learning, sensor networks, and multi-agent control. In this paper, we study D-OCO under unknown, time-and agent-varying feedback delays. While recent work has addressed this problem (Nguyen et al., 2024), existing algorithms assume prior knowledge of the total delay over agents and still suffer from suboptimal dependence on both the delay and network parameters. To overcome these limitations, we propose a novel algorithm that achieves an improved regret bound of O N $\sqrt$ d tot + N $\sqrt$ T (1-$\sigma$2) 1/4 , where T is the total horizon, d tot denotes the average total delay across agents, N is the number of agents, and 1 -$\sigma$ 2 is the spectral gap of the network. Our approach builds upon recent advances in D-OCO (Wan et al., 2024a), but crucially incorporates an adaptive learning rate mechanism via a decentralized communication protocol. This enables each agent to estimate delays locally using a gossip-based strategy without the prior knowledge of the total delay. We further extend our framework to the strongly convex setting and derive a sharper regret bound of O N $\delta$max ln T $\alpha$ , where $\alpha$ is the strong convexity parameter and $\delta$ max is the maximum number of missing observations averaged over agents. We also show that our upper bounds for both settings are tight up to logarithmic factors. Experimental results validate the effectiveness of our approach, showing improvements over existing benchmark algorithms.