Incorporating Group Prior into Variational Inference for Tail-User Behavior Modeling in CTR Prediction

TL;DR

This paper introduces GPSVI, a novel variational inference method that incorporates group priors and flow-based enhancements to better model tail-user preferences in CTR prediction, improving performance for users with sparse data.

Contribution

The paper proposes GPSVI, a new variational inference approach that integrates group priors and flow techniques to enhance tail-user behavior modeling in recommendation systems.

Findings

GPSVI improves tail-user CTR prediction accuracy.

The method maintains performance for head users.

Online A/B tests show significant system-wide benefits.

Abstract

User behavior modeling -- which aims to extract user interests from behavioral data -- has shown great power in Click-through rate (CTR) prediction, a key component in recommendation systems. Recently, attention-based algorithms have become a promising direction, as attention mechanisms emphasize the relevant interactions from rich behaviors. However, the methods struggle to capture the preferences of tail users with sparse interaction histories. To address the problem, we propose a novel variational inference approach, namely Group Prior Sampler Variational Inference (GPSVI), which introduces group preferences as priors to refine latent user interests for tail users. In GPSVI, the extent of adjustments depends on the estimated uncertainty of individual preference modeling. In addition, We further enhance the expressive power of variational inference by a volume-preserving flow. An appealing property of the GPSVI method is its ability to revert to traditional attention for head users with rich behavioral data while consistently enhancing performance for long-tail users with sparse behaviors. Rigorous analysis and extensive experiments demonstrate that GPSVI consistently improves the performance of tail users. Moreover, online A/B testing on a large-scale real-world recommender system further confirms the effectiveness of our proposed approach.