Distributed Online Learning for Joint Regret with Communication Constraints

TL;DR

This paper introduces a distributed online learning method that adapts to optimal graph partitions and efficiently manages communication constraints, achieving low joint regret in adversarial settings.

Contribution

It proposes a novel adaptive graph partitioning approach and a comparator-adaptive online convex optimization algorithm with delayed gradients.

Findings

The method adapts to the best graph partition among candidates.

It provides near-optimal gradient compression schemes based on communication bits.

The approach achieves low joint regret in adversarial online learning scenarios.

Abstract

We consider distributed online learning for joint regret with communication constraints. In this setting, there are multiple agents that are connected in a graph. Each round, an adversary first activates one of the agents to issue a prediction and provides a corresponding gradient, and then the agents are allowed to send a $b$-bit message to their neighbors in the graph. All agents cooperate to control the joint regret, which is the sum of the losses of the activated agents minus the losses evaluated at the best fixed common comparator parameters $u$. We observe that it is suboptimal for agents to wait for gradients that take too long to arrive. Instead, the graph should be partitioned into local clusters that communicate among themselves. Our main result is a new method that can adapt to the optimal graph partition for the adversarial activations and gradients, where the graph partition is selected from a set of candidate partitions. A crucial building block along the way is a new algorithm for online convex optimization with delayed gradient information that is comparator-adaptive, meaning that its joint regret scales with the norm of the comparator $||u||$. We further provide near-optimal gradient compression schemes depending on the ratio of $b$ and the dimension times the diameter of the graph.