Spotlight: Scalable Transport Layer Load Balancing for Data Center Networks
Ashkan Aghdai, Cing-Yu Chu, Yang Xu, David H. Dai, Jun Xu, H. Jonathan, Chao

TL;DR
Spotlight is a scalable, distributed load balancing system for data center networks that improves traffic distribution efficiency by using real-time load information and maintains connection consistency.
Contribution
It introduces a novel stateful flow dispatcher with distributed control and in-band dispatching, enabling scalable and efficient load balancing in data centers.
Findings
More efficient traffic distribution compared to existing methods
Achieves near-optimal service utilization
Operates effectively with low polling frequency (around 100ms)
Abstract
Load Balancing plays a vital role in modern data centers to distribute traffic among instances of network functions or services. State-of-the-art load balancers such as Silkroad dispatch traffic obliviously without considering the real-time utilization of service instances and therefore can lead to uneven load distribution and suboptimal performance. In this paper, we design and implement Spotlight, a scalable and distributed load balancing architecture that maintains connection-to-instance mapping consistency at the edge of data center networks. Spotlight uses a new stateful flow dispatcher which periodically polls instances' load and dispatches incoming connections to instances in proportion to their available capacity. Our design utilizes distributed control plane and in-band flow dispatching and thus scales horizontally in data center networks. Through extensive flow-level…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
