MP-Rec: Hardware-Software Co-Design to Enable Multi-Path Recommendation

TL;DR

This paper introduces MP-Rec, a hardware-software co-design approach that dynamically selects embedding representations and hardware platforms to significantly improve the performance and quality of deep learning recommendation systems.

Contribution

It proposes a hybrid embedding representation and a co-design technique that exploits hardware heterogeneity for optimized recommendation system performance.

Findings

Achieves 16.65x performance speedup with custom accelerators.

Improves prediction throughput by 2.49x and 3.76x on real hardware.

Enhances model quality with marginal accuracy improvements.

Abstract

Deep learning recommendation systems serve personalized content under diverse tail-latency targets and input-query loads. In order to do so, state-of-the-art recommendation models rely on terabyte-scale embedding tables to learn user preferences over large bodies of contents. The reliance on a fixed embedding representation of embedding tables not only imposes significant memory capacity and bandwidth requirements but also limits the scope of compatible system solutions. This paper challenges the assumption of fixed embedding representations by showing how synergies between embedding representations and hardware platforms can lead to improvements in both algorithmic- and system performance. Based on our characterization of various embedding representations, we propose a hybrid embedding representation that achieves higher quality embeddings at the cost of increased memory and compute requirements. To address the system performance challenges of the hybrid representation, we propose MP-Rec -- a co-design technique that exploits heterogeneity and dynamic selection of embedding representations and underlying hardware platforms. On real system hardware, we demonstrate how matching custom accelerators, i.e., GPUs, TPUs, and IPUs, with compatible embedding representations can lead to 16.65x performance speedup. Additionally, in query-serving scenarios, MP-Rec achieves 2.49x and 3.76x higher correct prediction throughput and 0.19% and 0.22% better model quality on a CPU-GPU system for the Kaggle and Terabyte datasets, respectively.