Personalized Tree-Based Progressive Regression Model for Watch-Time Prediction in Short Video Recommendation

TL;DR

This paper introduces PTPM, a personalized, data-driven tree-based regression model for more accurate watch-time prediction in short video recommendation, outperforming previous fixed-structure models.

Contribution

The paper proposes PTPM, a personalized, end-to-end learned decision tree model that adapts watch-time discretization to user data, improving prediction accuracy and efficiency.

Findings

PTPM outperforms state-of-the-art models in offline accuracy.

Online A/B tests show significant improvements in user engagement.

Fully deployed in core traffic, serving over 400 million users daily.

Abstract

In online video platforms, accurate watch time prediction has become a fundamental and challenging problem in video recommendation. Previous research has revealed that the accuracy of watch time prediction highly depends on both the transformation of watch-time labels and the decomposition of the estimation process. TPM (Tree based Progressive Regression Model) achieves State-of-the-Art performance with a carefully designed and effective decomposition paradigm. TPM discretizes the watch time into several ordinal intervals and organizes them into a binary decision tree, where each node corresponds to a specific interval. At each non-leaf node, a binary classifier is used to determine the specific interval in which the watch time variable most likely falls, based on the prediction outcome at its parent node. The tree structure is central to TPM, as it defines the decomposition of watch time estimation and how ordinal intervals are discretized. However, TPM uses a predefined full binary tree, which may be sub-optimal for two reasons. First, full binary trees imply equal partitioning of the watch time space, which may fail to capture the complexity of real-world distributions. Second, rather than relying on a fixed global structure, we advocate for a personalized, data-driven tree that can be learned end-to-end. Thus, we propose PTPM to enable highly personalized decomposition of watch estimation with better efficacy and efficiency. Moreover, we show that TPM suffers from selection bias due to conditional modeling and propose a simple solution. We conduct extensive experiments on offline datasets and online environments. Offline results show improved watch time accuracy, and online A/B tests further validate the effectiveness of our framework. PTPM has been fully deployed in core traffic scenarios and now serves over 400 million users daily.