BiCoLoR: Communication-Efficient Optimization with Bidirectional Compression and Local Training

TL;DR

BiCoLoR is a novel distributed optimization algorithm that combines bidirectional compression with local training, significantly reducing communication costs in federated learning over wireless networks.

Contribution

It introduces the first algorithm to integrate bidirectional compression with local training, providing accelerated guarantees in heterogeneous convex settings.

Findings

Outperforms existing algorithms in communication efficiency

Achieves accelerated convergence guarantees in convex settings

Establishes new standards for bidirectional communication compression

Abstract

Slow and costly communication is often the main bottleneck in distributed optimization, especially in federated learning where it occurs over wireless networks. We introduce BiCoLoR, a communication-efficient optimization algorithm that combines two widely used and effective strategies: local training, which increases computation between communication rounds, and compression, which encodes high-dimensional vectors into short bitstreams. While these mechanisms have been combined before, compression has typically been applied only to uplink (client-to-server) communication, leaving the downlink (server-to-client) side unaddressed. In practice, however, both directions are costly. We propose BiCoLoR, the first algorithm to combine local training with bidirectional compression using arbitrary unbiased compressors. This joint design achieves accelerated complexity guarantees in both convex and strongly convex heterogeneous settings. Empirically, BiCoLoR outperforms existing algorithms and establishes a new standard in communication efficiency.