DisaggRec: Architecting Disaggregated Systems for Large-Scale Personalized Recommendation

TL;DR

DisaggRec introduces a disaggregated system architecture for large-scale recommendation serving, significantly reducing total cost of ownership and improving resource utilization and system reliability in evolving datacenter environments.

Contribution

It presents DisaggRec, a novel disaggregated system that enables independent scaling of compute and memory, reducing TCO and enhancing flexibility for recommendation workloads.

Findings

Up to 49.3% TCO reduction with disaggregation.

DisaggRec achieves 21%-43.6% TCO savings with new hardware.

Resource idleness in monolithic servers wastes up to 30% TCO.

Abstract

Deep learning-based personalized recommendation systems are widely used for online user-facing services in production datacenters, where a large amount of hardware resources are procured and managed to reliably provide low-latency services without disruption. As the recommendation models continue to evolve and grow in size, our analysis projects that datacenters deployed with monolithic servers will spend up to 12.4x total cost of ownership (TCO) to meet the requirement of model size and complexity over the next three years. Moreover, through in-depth characterization, we reveal that the monolithic server-based cluster suffers resource idleness and wastes up to 30% TCO by provisioning resources in fixed proportions. To address this challenge, we propose DisaggRec, a disaggregated system for large-scale recommendation serving. DisaggRec achieves the independent decoupled scaling-out of the compute and memory resources to match the changing demands from fast-evolving workloads. It also improves system reliability by segregating the failures of compute nodes and memory nodes. These two main benefits from disaggregation collectively reduce the TCO by up to 49.3%. Furthermore, disaggregation enables flexible and agile provisioning of increasing hardware heterogeneity in future datacenters. By deploying new hardware featuring near-memory processing capability, our evaluation shows that the disaggregated cluster achieves 21%-43.6% TCO savings over the monolithic server-based cluster across a three-year span of model evolution.