NestPipe: Large-Scale Recommendation Training on 1,500+ Accelerators via Nested Pipelining

TL;DR

NestPipe is a scalable framework for recommendation model training that addresses both lookup and communication bottlenecks using nested pipelining, achieving significant speedups on large clusters.

Contribution

It introduces hierarchical sparse parallelism techniques, Dual-Buffer Pipelining and Frozen-Window Pipelining, to improve distributed training efficiency while maintaining synchronization.

Findings

Achieves up to 3.06x speedup on 1,536 workers.

Attains 94.07% scaling efficiency.

Effectively mitigates lookup and communication bottlenecks.

Abstract

Modern recommendation models have increased to trillions of parameters. As cluster scales expand to O(1k), distributed training bottlenecks shift from computation and memory to data movement, especially lookup and communication latency associated with embeddings. Existing solutions either optimize only one bottleneck or improve throughput by sacrificing training consistency. This paper presents NestPipe, a large-scale decentralized embedding training framework that tackles both bottlenecks while preserving synchronous training semantics. NestPipe exploits two hierarchical sparse parallelism opportunities through nested pipelining. At the inter-batch level, Dual-Buffer Pipelining (DBP) constructs a staleness-free five-stage pipeline through dual-buffer synchronization, mitigating lookup bottlenecks without embedding staleness. At the intra-batch level, we identify the embedding freezing phenomenon, which inspires Frozen-Window Pipelining (FWP) to overlap All2All communication with dense computation via coordinated stream scheduling and key-centric sample clustering. Experiments on production GPU and NPU clusters with 1,536 workers demonstrate that NestPipe achieves up to 3.06x speedup and 94.07% scaling efficiency.