Fair-Aurora: Comparing Fairness Strategies for Reinforcement Learning-Based Congestion Control in Multi-Flow Environments

TL;DR

This paper evaluates fairness strategies for Aurora, a deep RL congestion controller, demonstrating that modest reward shaping and loss-sensitivity tuning improve fairness and TCP-friendliness in multi-flow networks.

Contribution

It introduces and compares three post-hoc fairness strategies for Aurora, highlighting their effectiveness in multi-flow environments and dynamic scenarios.

Findings

Modest reward shaping achieves the best fairness with preserved throughput.

Loss-sensitivity tuning (Strategy C) is most TCP-friendly.

Observation augmentation (Strategy B) offers the greatest stability during flow changes.

Abstract

Reinforcement learning (RL) has emerged as a promising paradigm for Internet congestion control, achieving higher link utilization than classical heuristics. However, RL-based controllers trained in single-flow environments are not guaranteed to share bandwidth equitably when deployed in multi-flow networks. This paper investigates the fairness properties of Aurora~\cite{jay2019aurora}, a state-of-the-art deep RL congestion controller, and evaluates three post-hoc fairness strategies that preserve Aurora's RL architecture: \emph{reward shaping} (Strategy~A), \emph{observation augmentation} (Strategy~B), and \emph{loss-sensitivity tuning} (Strategy~C). Using a custom shared-bottleneck simulator and Jain's fairness index as the primary metric, we find that modest reward shaping achieves the best fairness while preserving aggregate throughput. All strategies maintain the total bandwidth budget with fairness being achieved through redistribution, not reduction. Beyond the 2-flow homogeneous setting, an extended evaluation across mixed Aurora--CUBIC competition and dynamic flow entry/exit scenarios shows that Strategy~C's loss-sensitivity emerges as the most TCP-friendly mechanism, while Strategy~B is the most stable through dynamic flow-set changes.