The Value of Personalized Recommendations: Evidence from Netflix

TL;DR

This paper develops a structural model to quantify the value of personalized Netflix recommendations, showing they significantly increase engagement and diversity, especially for mid-popularity content.

Contribution

It introduces a novel discrete choice model that isolates the utility of recommendations and evaluates counterfactual scenarios for Netflix's system.

Findings

Replacing Netflix's recommender with simpler algorithms reduces engagement by up to 12%.

Most engagement gains come from effective targeting rather than mere exposure.

Recommendations notably boost consumption of mid-popularity content.

Abstract

Personalized recommendation systems shape much of user choice online, yet their targeted nature makes separating out the value of recommendation and the underlying goods challenging. We build a discrete choice model that embeds recommendation-induced utility, low-rank heterogeneity, and flexible state dependence and apply the model to viewership data at Netflix. We exploit idiosyncratic variation introduced by the recommendation algorithm to identify and separately value these components as well as to recover model-free diversion ratios that we can use to validate our structural model. We use the model to evaluate counterfactuals that quantify the incremental engagement generated by personalized recommendations. First, we show that replacing the current recommender system with a matrix factorization or popularity-based algorithm would lead to 4% and 12% reduction in engagement, respectively, and decreased consumption diversity. Second, most of the consumption increase from recommendations comes from effective targeting, not mechanical exposure, with the largest gains for mid-popularity goods (as opposed to broadly appealing or very niche goods).