DiffFlow: Differentiating Short and Long Flows for Load Balancing in Data Center Networks

TL;DR

DiffFlow is a load balancing method for data center networks that differentiates short and long flows, using SDN to apply RPS to long flows and ECMP to short flows, improving network performance.

Contribution

It introduces a novel load balancing approach that detects long flows and applies RPS, while using ECMP for short flows, enhancing throughput and reducing hot-spots.

Findings

Outperforms individual RPS or ECMP in throughput

Maintains throughput levels comparable to RPS

Reduces network hot-spots and improves load balancing

Abstract

In current Data Center Networks (DCNs), Equal- Cost MultiPath (ECMP) is used as the de-facto routing protocol. However, ECMP does not differentiate between short and long flows, the two main categories of flows depending on their duration (lifetime). This issue causes hot-spots in the network, affecting negatively the Flow Completion Time (FCT) and the throughput, the two key performance metrics in data center networks. Previous work on load balancing proposed solutions such as splitting long flows into short flows, using per-packet forwarding approaches, and isolating the paths of short and long flows. We propose DiffFlow, a new load balancing solution which detects long flows and forwards packets using Random Packet Spraying (RPS) with help of SDN, whereas the flows with small duration are forwarded with ECMP by default. The use of ECMP for short flows is reasonable, as it does not create the out-of-order problem; at the same time, RPS for long flows can efficiently help to load balancing the entire network, given that long flows represent most of the traffic in DCNs. The results show that our DiffFlow solution outperforms both the individual usage of either RPS or ECMP, while the overall throughput achieved is maintained at the level comparable to RPS.