Estimating Incremental Acquisition of Content Launches in a Subscription Service

TL;DR

This paper introduces a scalable, non-experimental methodology to estimate the causal impact of content launches on new subscriber acquisition in subscription services, accounting for confounding factors and enabling attribution at the subscriber level.

Contribution

It proposes a novel approach that estimates incremental acquisition impact using consumption data and adjusts for various confounders, without requiring randomized experiments.

Findings

Provides top-line impact estimates at content/day/region level

Develops an algorithm for subscriber-level attribution

Includes validation methods for impact plausibility

Abstract

Subscription services face a difficult problem when estimating the causal impact of content launches on acquisition. Customers buy subscriptions, not individual pieces of content, and once subscribed they may consume many pieces of content in addition to the one(s) that drew them to the service. In this paper, we propose a scalable methodology to estimate the incremental acquisition impact of content launches in a subscription business model when randomized experimentation is not feasible. Our approach uses simple assumptions to transform the problem into an equivalent question: what is the expected consumption rate for new subscribers who did not join due to the content launch? We estimate this counterfactual rate using the consumption rate of new subscribers who joined just prior to launch, while making adjustments for variation related to subscriber attributes, the in-product experience, and seasonality. We then compare our counterfactual consumption to the actual rate in order to back out an acquisition estimate. Our methodology provides top-line impact estimates at the content / day / region grain. Additionally, to enable subscriber-level attribution, we present an algorithm that assigns specific individual accounts to add up to the top-line estimate. Subscriber-level attribution is derived by solving an optimization problem to minimize the number of subscribers attributed to more than one piece of content, while maximizing the average propensity to be incremental for subscribers attributed to each piece of content. Finally, in the absence of definitive ground truth, we present several validation methods which can be used to assess the plausibility of impact estimates generated by these methods.