Ranking Across Different Content Types: The Robust Beauty of Multinomial Blending

TL;DR

This paper introduces multinomial blending, a simple and versatile method for ranking diverse content types across media platforms, improving ranking quality and stability in dynamic environments.

Contribution

It proposes multinomial blending, a novel approach compatible with existing ranking algorithms, enhancing cross-content-type ranking performance and interpretability.

Findings

MB improves ranking quality over baselines

MB demonstrates stability in dynamic user environments

A/B testing confirms effectiveness in Amazon Music

Abstract

An increasing number of media streaming services have expanded their offerings to include entities of multiple content types. For instance, audio streaming services that started by offering music only, now also offer podcasts, merchandise items, and videos. Ranking items across different content types into a single slate poses a significant challenge for traditional learning-to-rank (LTR) algorithms due to differing user engagement patterns for different content types. We explore a simple method for cross-content-type ranking, called multinomial blending (MB), which can be used in conjunction with most existing LTR algorithms. We compare MB to existing baselines not only in terms of ranking quality but also from other industry-relevant perspectives such as interpretability, ease-of-use, and stability in dynamic environments with changing user behavior and ranking model retraining. Finally, we report the results of an A/B test from an Amazon Music ranking use-case.