RNG: Flat Datacenter Networks at Scale

TL;DR

RNG introduces a scalable, cost-effective flat datacenter network based on quasi-random graphs, featuring a novel routing protocol and passive optical cabling, outperforming traditional fat trees in performance and cost.

Contribution

The paper presents the first practical implementation of flat datacenter networks using RNG, with a new distributed routing protocol and passive optical cabling, enabling scalable, fault-tolerant, and cost-efficient networks.

Findings

RNG matches or exceeds fat tree performance across various traffic patterns.

RNG reduces costs by up to 45% compared to traditional fat trees.

RNG is adopted as the default network at Amazon for most workloads.

Abstract

We design and deploy in production the first flat datacenter networks. Our design, called RNG, is based on quasi-random graphs. While the cost and fault-tolerance benefits of such topologies have been long known, their practical realization has been hampered by a lack of scalable routing and cabling approaches. RNG has a new distributed routing protocol that exploits the properties of random graphs to find a large number of edge disjoint paths between pairs of endpoints. It uses a novel passive optical device that internally shuffles cables, which makes its cabling complexity similar to that of fat trees. We show that RNG matches or exceeds the performance of fat trees for a range of traffic patterns, despite being up to 45% cheaper. RNG is now the default datacenter network for most workloads at Amazon.