WWW.Serve: Interconnecting Global LLM Services through Decentralization

TL;DR

WWW.Serve introduces a decentralized framework for global LLM service interconnection, enabling flexible participation and autonomous request dispatch, significantly improving service levels and latency without centralized control.

Contribution

It proposes a novel decentralized LLM serving framework that supports flexible policies and self-organizing request allocation, overcoming limitations of existing centralized or rigid systems.

Findings

Improves global SLO attainment by up to 1.5x

Reduces latency by 27.6%

Performance approaches or surpasses centralized scheduling

Abstract

Large language model (LLM) services are mostly centralized, leading to scalability bottlenecks and underutilization of substantial scattered GPU resources. While decentralization offers a promising alternative, existing frameworks primarily focus on cooperation among GPU providers while overlooking their inherent competitive dynamics, imposing substantial constraints such as excessive platform-level oversight or rigid requirements to execute all assigned requests using fixed software stacks on fixed hardware configurations. We argue that such assumptions are unrealistic in real-world decentralized environments. To this end, we propose WWW$.$Serve, a decentralized framework for interconnecting LLM services worldwide. It allows participants to flexibly determine their participation policies and resource commitments, and supports self-organizing request dispatch, enabling the network to autonomously allocate requests without centralized coordination. Empirically, we show that WWW$.$Serve improves global SLO (service-level-objective) attainment by up to 1.5x and lowers latency by 27.6%. Its performance approaches, and in some cases surpasses, centralized scheduling, while fully preserving the benefits of decentralization. These results highlight WWW$.$Serve as a promising foundation for real-world, decentralized LLM serving.