Can we predict the Most Replayed data of video streaming platforms?

TL;DR

This paper investigates the feasibility of predicting the Most Replayed segments in YouTube videos using deep learning, introducing a new dataset and demonstrating that DL models outperform humans and random guesses.

Contribution

The authors create the YTMR500 dataset for MR prediction and evaluate DL models, showing they outperform humans and random predictions, highlighting the task's difficulty and potential for improvement.

Findings

DL models outperform human accuracy in MR prediction

All evaluated models surpass random guessing

The task remains challenging but promising for future research

Abstract

Predicting which specific parts of a video users will replay is important for several applications, including targeted advertisement placement on video platforms and assisting video creators. In this work, we explore whether it is possible to predict the Most Replayed (MR) data from YouTube videos. To this end, we curate a large video benchmark, the YTMR500 dataset, which comprises 500 YouTube videos with MR data annotations. We evaluate Deep Learning (DL) models of varying complexity on our dataset and perform an extensive ablation study. In addition, we conduct a user study to estimate the human performance on MR data prediction. Our results show that, although by a narrow margin, all the evaluated DL models outperform random predictions. Additionally, they exceed human-level accuracy. This suggests that predicting the MR data is a difficult task that can be enhanced through the assistance of DL. Finally, we believe that DL performance on MR data prediction can be further improved, for example, by using multi-modal learning. We encourage the research community to use our benchmark dataset to further investigate automatic MR data prediction.