Learning to Harness Bandwidth with Multipath Congestion Control and Scheduling

TL;DR

This paper introduces a Deep Q-Learning framework for joint congestion control and packet scheduling in MPTCP, significantly improving performance and adaptability over existing algorithms.

Contribution

It presents a novel DQL-based approach for MPTCP that learns optimal control and scheduling policies using neural networks with stability analysis.

Findings

Outperforms existing MPTCP algorithms like LIA, OLIA, BALIA

Demonstrates robustness to changing network conditions

Provides dynamic path exploration and exploitation

Abstract

Multipath TCP (MPTCP) has emerged as a facilitator for harnessing and pooling available bandwidth in wireless/wireline communication networks and in data centers. Existing implementations of MPTCP such as, Linked Increase Algorithm (LIA), Opportunistic LIA (OLIA) and BAlanced LInked Adaptation (BALIA) include separate algorithms for congestion control and packet scheduling, with pre-selected control parameters. We propose a Deep Q-Learning (DQL) based framework for joint congestion control and packet scheduling for MPTCP. At the heart of the solution is an intelligent agent for interface, learning and actuation, which learns from experience optimal congestion control and scheduling mechanism using DQL techniques with policy gradients. We provide a rigorous stability analysis of system dynamics which provides important practical design insights. In addition, the proposed DQL-MPTCP algorithm utilizes the `recurrent neural network' and integrates it with `long short-term memory' for continuously i) learning dynamic behavior of subflows (paths) and ii) responding promptly to their behavior using prioritized experience replay. With extensive emulations, we show that the proposed DQL-based MPTCP algorithm outperforms MPTCP LIA, OLIA and BALIA algorithms. Moreover, the DQL-MPTCP algorithm is robust to time-varying network characteristics, and provides dynamic exploration and exploitation of paths.