CALF: Communication-Aware Learning Framework for Distributed Reinforcement Learning

TL;DR

CALF is a novel training framework for distributed reinforcement learning that incorporates realistic network conditions during simulation, significantly improving deployment performance across heterogeneous hardware in real-world scenarios.

Contribution

This paper introduces CALF, a framework that models network constraints during training to enhance the robustness of distributed RL policies in real-world network environments.

Findings

Network-aware training reduces deployment performance gaps.

Explicitly modeling communication constraints improves real-world robustness.

Network conditions are a key factor in sim-to-real transfer for distributed RL.

Abstract

Distributed reinforcement learning policies face network delays, jitter, and packet loss when deployed across edge devices and cloud servers. Standard RL training assumes zero-latency interaction, causing severe performance degradation under realistic network conditions. We introduce CALF (Communication-Aware Learning Framework), which trains policies under realistic network models during simulation. Systematic experiments demonstrate that network-aware training substantially reduces deployment performance gaps compared to network-agnostic baselines. Distributed policy deployments across heterogeneous hardware validate that explicitly modelling communication constraints during training enables robust real-world execution. These findings establish network conditions as a major axis of sim-to-real transfer for Wi-Fi-like distributed deployments, complementing physics and visual domain randomisation.