Serendipity and strategy in rapid innovation

TL;DR

This paper models innovation as a search process in a design space, revealing how serendipity and strategy are interconnected through the changing importance of components, supported by historical data analysis.

Contribution

It unifies serendipitous and strategic views of innovation by analyzing the mathematics of component usefulness over time and validating with real-world data.

Findings

Component usefulness crossovers are often unanticipated, indicating serendipity.

Predictable crossovers can be exploited strategically to accelerate innovation.

Changing component importance links serendipity and strategy in innovation processes.

Abstract

Innovation is to organizations what evolution is to organisms: it is how organisations adapt to changes in the environment and improve. Governments, institutions and firms that innovate are more likely to prosper and stand the test of time; those that fail to do so fall behind their competitors and succumb to market and environmental change. Yet despite steady advances in our understanding of evolution, what drives innovation remains elusive. On the one hand, organizations invest heavily in systematic strategies to drive innovation. On the other, historical analysis and individual experience suggest that serendipity plays a significant role in the discovery process. To unify these two perspectives, we analyzed the mathematics of innovation as a search process for viable designs across a universe of building blocks. We then tested our insights using historical data from language, gastronomy and technology. By measuring the number of makeable designs as we acquire more components, we observed that the relative usefulness of different components is not fixed, but cross each other over time. When these crossovers are unanticipated, they appear to be the result of serendipity. But when we can predict crossovers ahead of time, they offer an opportunity to strategically increase the growth of our product space. Thus we find that the serendipitous and strategic visions of innovation can be viewed as different manifestations of the same thing: the changing importance of component building blocks over time.