It's the People, Not the Placement: Rethinking Allocations in Post-Moore Clouds

TL;DR

This paper advocates for a paradigm shift in resource management in modern heterogeneous cloud environments, emphasizing dynamic allocation and new interaction architectures to improve efficiency and reduce costs.

Contribution

It introduces a novel approach to cloud resource management that replaces static allocations with dynamic, renegotiable resource sharing tailored for heterogeneous hardware.

Findings

Dynamic allocation improves resource utilization.

Cost reductions achieved through continuous renegotiation.

New architecture enhances operator-tenant-application interactions.

Abstract

The Cambrian explosion of new accelerators, driven by the slowdown of Moore's Law, has created significant resource management challenges for modern IaaS clouds. Unlike the homogeneous datacenters backing legacy clouds, emerging neoclouds amass a diverse portfolio of heterogeneous hardware -- NVIDIA GPUs, TPUs, Trainium chips, and FPGAs. Neocloud operators and tenants must transition from managing a single large pool of computational resources to navigating a set of highly fragmented and constrained pools. We argue that cloud resource management mechanisms and interfaces require a fundamental rethink to enable efficient and economical neoclouds. Specifically we propose shifting from long-term static resource allocation with fixed-pricing to dynamic allocation with continuous, multilateral cost re-negotatiaton. We demonstrate this approach is not only feasible for modern applications but also significantly improves resource efficiency and reduces costs. Finally, we propose a new architecture for the interaction between operators, tenants, and applications in neoclouds.