2BRobust -- Overcoming TCP BBR Performance Degradation in Virtual Machines under CPU Contention

TL;DR

This paper investigates the performance degradation of TCP BBR congestion control in virtual machines under CPU contention, revealing significant throughput issues and proposing a patch to improve robustness.

Contribution

It develops a framework to measure TCP throughput under CPU contention, characterizes BBR's performance issues, and proposes a minimal patch to mitigate throughput degradation in virtualized environments.

Findings

BBR throughput drops below 20 Mbps during CPU contention

The performance degradation is consistent across hypervisors and BDP conditions

A simple patch effectively restores BBR throughput in critical scenarios

Abstract

Motivated by the recent introduction and large-scale deployment of BBR congestion control algorithms, multiple studies have investigated the performance and fairness implications of this shift from loss-based to delay-based congestion control. Given the potential Internet-wide adoption of BBR, we must also consider its robustness in network and system scenarios. One such scenario is Cloud-based Virtual Machine (VM) networking - highly relevant in today's CDN-centric Internet. Interestingly, previous work has shown significant performance problems of BBRv1-2 running in Xen VMs, with BBR performance dropping to almost zero when CPU credit is low. In this paper, we develop a framework for measuring TCP throughput under fully controlled CPU contention, which uses Linux deadline scheduling to emulate generalized CPU contention conditions. Our measurements reveal that - in stark contrast to Cubic! - BBR throughput can break down during CPU contention under any hypervisor and all tested BDP conditions. Characterizing this performance degradation on a fine-granular level, we show that CPU limited BBR senders are capped at very low throughput levels below 10-20 Mbps. This finding implies that an Internet-wide shift from Cubic to BBR could harm the Internet's overall robustness, if not deployed with caution. To detect and overcome CPU-limited throughput, we propose a minimal BBR patch which detects the problematic situation by monitoring inflight bytes and reacts by increasing the pacing rate to make better use of the available CPU time. We show that our BBR patch overcomes the throughput problem for the most critical cases.