Beyond Views: Measuring and Predicting Engagement in Online Videos

TL;DR

This paper introduces a large-scale measurement of online video engagement, proposing new metrics and demonstrating their stability and predictability, which can improve content creation and recommendation strategies.

Contribution

It presents the first large-scale analysis of video engagement metrics, introduces a new relative engagement metric, and shows these metrics are stable and predictable from video context.

Findings

Engagement metrics are stable over time.

A new relative engagement metric correlates with perceived quality.

Engagement can be predicted with high accuracy from video context.

Abstract

The share of videos in the internet traffic has been growing, therefore understanding how videos capture attention on a global scale is also of growing importance. Most current research focus on modeling the number of views, but we argue that video engagement, or time spent watching is a more appropriate measure for resource allocation problems in attention, networking, and promotion activities. In this paper, we present a first large-scale measurement of video-level aggregate engagement from publicly available data streams, on a collection of 5.3 million YouTube videos published over two months in 2016. We study a set of metrics including time and the average percentage of a video watched. We define a new metric, relative engagement, that is calibrated against video properties and strongly correlate with recognized notions of quality. Moreover, we find that engagement measures of a video are stable over time, thus separating the concerns for modeling engagement and those for popularity -- the latter is known to be unstable over time and driven by external promotions. We also find engagement metrics predictable from a cold-start setup, having most of its variance explained by video context, topics and channel information -- R2=0.77. Our observations imply several prospective uses of engagement metrics -- choosing engaging topics for video production, or promoting engaging videos in recommender systems.