LiNR: Model Based Neural Retrieval on GPUs at LinkedIn

TL;DR

LiNR is a GPU-based, large-scale retrieval system at LinkedIn that integrates models and indexes for efficient, scalable, and differentiable search, leading to improved user engagement.

Contribution

The paper presents LiNR, a novel GPU-based retrieval system supporting billion-sized indexes with integrated models, attribute pre-filtering, and scalable training methods.

Findings

Achieved a 3% increase in daily active users.

Supported billion-sized indexes on GPUs.

Enabled end-to-end differentiable retrieval and ranking.

Abstract

This paper introduces LiNR, LinkedIn's large-scale, GPU-based retrieval system. LiNR supports a billion-sized index on GPU models. We discuss our experiences and challenges in creating scalable, differentiable search indexes using TensorFlow and PyTorch at production scale. In LiNR, both items and model weights are integrated into the model binary. Viewing index construction as a form of model training, we describe scaling our system for large indexes, incorporating full scans and efficient filtering. A key focus is on enabling attribute-based pre-filtering for exhaustive GPU searches, addressing the common challenge of post-filtering in KNN searches that often reduces system quality. We further provide multi-embedding retrieval algorithms and strategies for tackling cold start issues in retrieval. Our advancements in supporting larger indexes through quantization are also discussed. We believe LiNR represents one of the industry's first Live-updated model-based retrieval indexes. Applied to out-of-network post recommendations on LinkedIn Feed, LiNR has contributed to a 3% relative increase in professional daily active users. We envisage LiNR as a step towards integrating retrieval and ranking into a single GPU model, simplifying complex infrastructures and enabling end-to-end optimization of the entire differentiable infrastructure through gradient descent.