High Throughput Data Center Topology Design

TL;DR

This paper establishes an upper bound on data center network throughput with identical switches, shows random graphs perform near this bound, and demonstrates how heterogeneous switch configurations can significantly improve throughput.

Contribution

It introduces the first non-trivial throughput upper bound for homogeneous networks and explores heterogeneous topologies to enhance data center performance.

Findings

Random graphs achieve throughput within a few percent of the theoretical maximum.

Heterogeneous switch configurations can increase throughput by up to 43%.

Homogeneous topologies may be nearing their optimal capacity.

Abstract

With high throughput networks acquiring a crucial role in supporting data-intensive applications, a variety of data center network topologies have been proposed to achieve high capacity at low cost. While this literature explores a large number of design points, even in the limited case of a network of identical switches, no proposal has been able to claim any notion of optimality. The case of heterogeneous networks, incorporating multiple line-speeds and port-counts as data centers grow over time, introduces even greater complexity. In this paper, we present the first non-trivial upper-bound on network throughput under uniform traffic patterns for any topology with identical switches. We then show that random graphs achieve throughput surprisingly close to this bound, within a few percent at the scale of a few thousand servers. Apart from demonstrating that homogeneous topology design may be reaching its limits, this result also motivates our use of random graphs as building blocks to explore the design of heterogeneous networks. Given a heterogeneous pool of network switches, through experiments and analysis, we explore how the distribution of servers across switches and the interconnection of switches affect network throughput. We apply these insights to a real-world heterogeneous data center topology, VL2, demonstrating as much as 43% higher throughput with the same equipment.