Tweedie Regression for Video Recommendation System

TL;DR

This paper introduces Tweedie Regression for video recommendation systems, shifting from classification to regression to optimize user viewing time and revenue, demonstrating significant improvements through offline and online tests.

Contribution

It proposes using Tweedie Loss for recommendation regression tasks, providing theoretical insights and a framework for designing loss functions aligned with business objectives.

Findings

Enhanced user engagement and revenue in online A/B tests

Tweedie Loss outperforms traditional loss functions in capturing user interests

Theoretical analysis shows advantages of Tweedie Regression over Logloss

Abstract

Modern recommendation systems aim to increase click-through rates (CTR) for better user experience, through commonly treating ranking as a classification task focused on predicting CTR. However, there is a gap between this method and the actual objectives of businesses across different sectors. In video recommendation services, the objective of video on demand (VOD) extends beyond merely encouraging clicks, but also guiding users to discover their true interests, leading to increased watch time. And longer users watch time will leads to more revenue through increased chances of presenting online display advertisements. This research addresses the issue by redefining the problem from classification to regression, with a focus on maximizing revenue through user viewing time. Due to the lack of positive labels on recommendation, the study introduces Tweedie Loss Function, which is better suited in this scenario than the traditional mean square error loss. The paper also provides insights on how Tweedie process capture users diverse interests. Our offline simulation and online A/B test revealed that we can substantially enhance our core business objectives: user engagement in terms of viewing time and, consequently, revenue. Additionally, we provide a theoretical comparison between the Tweedie Loss and the commonly employed viewing time weighted Logloss, highlighting why Tweedie Regression stands out as an efficient solution. We further outline a framework for designing a loss function that focuses on a singular objective.