Real-time Indexing for Large-scale Recommendation by Streaming Vector Quantization Retriever

TL;DR

This paper introduces a real-time streaming vector quantization index that improves large-scale recommendation retrieval by enabling immediate, balanced, and reparable indexing, supporting complex ranking models and enhancing user engagement.

Contribution

The paper presents a novel streaming vector quantization index structure that operates in real time, balancing and repairing itself, and supports complex ranking models, addressing fundamental index effectiveness issues.

Findings

Deployed in Douyin and Douyin Lite, replacing existing retrievers.

Achieved significant user engagement improvements.

Supports complex ranking models with real-time indexing.

Abstract

Retrievers, which form one of the most important recommendation stages, are responsible for efficiently selecting possible positive samples to the later stages under strict latency limitations. Because of this, large-scale systems always rely on approximate calculations and indexes to roughly shrink candidate scale, with a simple ranking model. Considering simple models lack the ability to produce precise predictions, most of the existing methods mainly focus on incorporating complicated ranking models. However, another fundamental problem of index effectiveness remains unresolved, which also bottlenecks complication. In this paper, we propose a novel index structure: streaming Vector Quantization model, as a new generation of retrieval paradigm. Streaming VQ attaches items with indexes in real time, granting it immediacy. Moreover, through meticulous verification of possible variants, it achieves additional benefits like index balancing and reparability, enabling it to support complicated ranking models as existing approaches. As a lightweight and implementation-friendly architecture, streaming VQ has been deployed and replaced all major retrievers in Douyin and Douyin Lite, resulting in remarkable user engagement gain.