Listen to Users, but Only 85% of the Time: How Black Swans Can Save Innovation in a Data-Driven World

TL;DR

This paper proposes leveraging Black Swan events in digital design by balancing data-driven experimentation with unpredictable, innovative 'crazy' designs, using Taleb's barbell strategy to foster innovation.

Contribution

It introduces the concept of Black Swan designs in digital innovation and suggests a practical resource allocation strategy to balance data-driven and radical innovation.

Findings

Online experimentation is a fat-tailed phenomenon prone to Black Swans.

Black Swan designs are 'crazy' but can lead to breakthrough innovations.

Applying the barbell strategy can enhance innovation while maintaining data-driven reliability.

Abstract

Data-driven design is a proven success factor that more and more digital businesses embrace. At the same time, academics and practitioners alike warn that when virtually everything must be tested and proven with numbers, that can stifle creativity and innovation. This article argues that Taleb's Black Swan theory can solve this dilemma. It shows that online experimentation, and therefore digital design, are fat-tailed phenomena and, hence, prone to Black Swans. It introduces the notion of Black Swan designs -- "crazy" designs that make sense only in hindsight -- along with four specific criteria. To ensure incremental improvements and their potential for innovation, businesses should apply Taleb's barbell strategy: Invest 85-90% of resources into data-driven approaches and 10-15% into potential Black Swans.