PET: Multi-agent Independent PPO-based Automatic ECN Tuning for High-Speed Data Center Networks

TL;DR

This paper introduces PET, a multi-agent learning-based scheme that dynamically tunes ECN thresholds in high-speed data center networks, significantly improving performance over static and existing adaptive methods.

Contribution

PET is a novel multi-agent independent PPO-based scheme that adaptively adjusts ECN thresholds considering multiple network factors, enhancing congestion control in data centers.

Findings

PET outperforms static schemes in flow completion time.

PET achieves faster convergence and better robustness.

PET maintains queue length stability under dynamic traffic.

Abstract

Explicit Congestion Notification (ECN)-based congestion control schemes have been widely adopted in high-speed data center networks (DCNs), where the ECN marking threshold plays a determinant role in guaranteeing a packet lossless DCN. However, existing approaches either employ static settings with immutable thresholds that cannot be dynamically self-adjusted to adapt to network dynamics, or fail to take into account many-to-one traffic patterns and different requirements of different types of traffic, resulting in relatively poor performance. To address these problems, this paper proposes a novel learning-based automatic ECN tuning scheme, named PET, based on the multi-agent Independent Proximal Policy Optimization (IPPO) algorithm. PET dynamically adjusts ECN thresholds by fully considering pivotal congestion-contributing factors, including queue length, output data rate, output rate of ECN-marked packets, current ECN threshold, the extent of incast, and the ratio of mice and elephant flows. PET adopts the Decentralized Training and Decentralized Execution (DTDE) paradigm and combines offline and online training to accommodate network dynamics. PET is also fair and readily deployable with commodity hardware. Comprehensive experimental results demonstrate that, compared with state-of-the-art static schemes and the learning-based automatic scheme, our PET achieves better performance in terms of flow completion time, convergence rate, queue length variance, and system robustness.