Sequential Regression for Continuous Value Prediction using Residual Quantization

TL;DR

This paper introduces a residual quantization-based sequence learning framework for continuous value prediction in recommendation systems, addressing data complexity and distribution challenges more effectively than existing methods.

Contribution

It proposes a novel residual quantization approach with a representation learning objective that improves prediction accuracy and generalization in continuous value tasks.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Demonstrates strong generalization across diverse tasks.

Shows effectiveness in large-scale online industrial experiments.

Abstract

Continuous value prediction plays a crucial role in industrial-scale recommendation systems, including tasks such as predicting users' watch-time and estimating the gross merchandise value (GMV) in e-commerce transactions. However, it remains challenging due to the highly complex and long-tailed nature of the data distributions. Existing generative approaches rely on rigid parametric distribution assumptions, which fundamentally limits their performance when such assumptions misalign with real-world data. Overly simplified forms cannot adequately model real-world complexities, while more intricate assumptions often suffer from poor scalability and generalization. To address these challenges, we propose a residual quantization (RQ)-based sequence learning framework that represents target continuous values as a sum of ordered quantization codes, predicted recursively from coarse to fine granularity with diminishing quantization errors. We introduce a representation learning objective that aligns RQ code embedding space with the ordinal structure of target values, allowing the model to capture continuous representations for quantization codes and further improving prediction accuracy. We perform extensive evaluations on public benchmarks for lifetime value (LTV) and watch-time prediction, alongside a large-scale online experiment for GMV prediction on an industrial short-video recommendation platform. The results consistently show that our approach outperforms state-of-the-art methods, while demonstrating strong generalization across diverse continuous value prediction tasks in recommendation systems.