Bring Your Own Objective: Inter-operability of Network Objectives in Datacenters

TL;DR

DMart introduces a decentralized, auction-based scheduling framework for datacenter networks that allows multiple, diverse performance objectives to coexist and optimize simultaneously, improving overall network efficiency.

Contribution

This paper presents DMart, a novel decentralized auction-based scheduling system enabling multiple network objectives to coexist in datacenter fabrics without centralized control.

Findings

DMart matches specialized schedulers' performance on their metrics.

DMart reduces deadline misses by 2x compared to existing solutions.

DMart decreases coflow completion times by 1.6x.

Abstract

Datacenter networks are currently locked in a "tyranny of the single objective". While modern workloads demand diverse performance goals, ranging from coflow completion times, per-flow fairness, short-flow latencies, existing fabrics are typically hardcoded for a single metric. This rigid coupling ensures peak performance when application and network objectives align, but results in abysmal performance when they diverge. We propose DMart, a decentralized scheduling framework that treats network bandwidth as a competitive marketplace. In DMart, applications independently encode the urgency and importance of their network traffic into autonomous bids, allowing diverse objectives to co-exist natively on the same fabric. To meet the extreme scale and sub-microsecond requirements of modern datacenters, DMart implements distributed, per-link, per-RTT auctions, without relying on ILPs, centralized schedulers, or complex priority queues. We evaluate DMart using packet-level simulations and compare it against network schedulers designed for individual metrics, e.g., pFabric and Sincronia. DMart matches the performance of specialized schedulers on their own "home turf" while simultaneously optimizing secondary metrics. Compared to pFabric and Sincronia, DMart reduces deadline misses by 2x and coflow completion times by 1.6x respectively, while matching pFabric short-flow completion times.