Your MMM is Broken: Identification of Nonlinear and Time-varying Effects in Marketing Mix Models

TL;DR

This paper demonstrates that nonlinear and time-varying effects in marketing mix models are often not distinguishable from simpler dynamic models using standard data, leading to potential misinterpretations of marketing effectiveness.

Contribution

It introduces a Bayesian nonparametric model to analyze the identifiability issues and proposes experimental strategies to differentiate effects in marketing mix modeling.

Findings

Nonlinear and time-varying effects are often conflated in standard data.

Autocorrelated variables increase the likelihood of conflation.

Experimental design can help distinguish between effects.

Abstract

Recent years have seen a resurgence in interest in marketing mix models (MMMs), which are aggregate-level models of marketing effectiveness. Often these models incorporate nonlinear effects, and either implicitly or explicitly assume that marketing effectiveness varies over time. In this paper, we show that nonlinear and time-varying effects are often not identifiable from standard marketing mix data: while certain data patterns may be suggestive of nonlinear effects, such patterns may also emerge under simpler models that incorporate dynamics in marketing effectiveness. This lack of identification is problematic because nonlinearities and dynamics suggest fundamentally different optimal marketing allocations. We examine this identification issue through theory and simulations, wherein we explore the exact conditions under which conflation between the two types of models is likely to occur. In doing so, we introduce a flexible Bayesian nonparametric model that allows us to both flexibly simulate and estimate different data-generating processes. We show that conflating the two types of effects is especially likely in the presence of autocorrelated marketing variables, which are common in practice, especially given the widespread use of stock variables to capture long-run effects of advertising. We illustrate these ideas through numerous empirical applications to real-world marketing mix data, showing the prevalence of the conflation issue in practice. Finally, we show how marketers can avoid this conflation, by designing experiments that strategically manipulate spending in ways that pin down model form.