A Deep Reinforcement Learning-Based TCP Congestion Control Algorithm: Design, Simulation, and Evaluation

TL;DR

This paper presents a novel DRL-based TCP congestion control algorithm that dynamically adjusts the congestion window, achieving better latency and buffer management while maintaining throughput, as demonstrated through NS-3 simulations.

Contribution

Introduces a Deep Reinforcement Learning-based TCP congestion control method using DQNs, showing improved latency and buffer management over traditional algorithms.

Findings

46.29% RTT reduction compared to TCP Cubic

Near-zero packet loss with the DRL algorithm

Comparable throughput to traditional TCP algorithms

Abstract

This paper introduces a Deep Reinforcement Learning (DRL) based TCP congestion-control algorithm that uses a Deep Q-Network (DQN) to adapt the congestion window (cWnd) dynamically based on observed network state. The proposed approach utilizes DQNs to optimize the congestion window by observing key network parameters and taking real-time actions. The algorithm is trained and evaluated within the NS-3 network simulator using the OpenGym interface. The results demonstrate that the DRL-based algorithm provides a superior balance between throughput and latency compared to both traditional TCP New Reno and TCP Cubic algorithms. Specifically: Compared to TCP Cubic, the DRL algorithm achieved comparable throughput (statistically insignificant difference of -3.79%, $p>0.05$) while delivering a massive 46.29% reduction in Round-Trip Time (RTT). Furthermore, the DRL agent maintained near-zero packet loss, whereas Cubic suffered from significant buffer overflow. Compared to TCP New Reno, the DRL algorithm achieved comparable throughput (+0.38%) with a 32.40% reduction in RTT. Results from NS-3 simulations indicate that the proposed DRL agent effectively mitigates bufferbloat without compromising bandwidth utilization. This study emphasizes the potential of reinforcement learning techniques for solving complex congestion control problems in modern networks by learning the network capacity rather than saturating it.