Nothing Deceives Like Success: Social Learning and the Illusion of Understanding in Science

TL;DR

This paper uses agent-based simulations to show that success-driven social learning in science can create an illusion of understanding, leading to overconfidence, reduced exploration, and paradoxically worse outcomes.

Contribution

It demonstrates that success bias in scientific communities can hinder discovery and increase inequality, especially in complex problem environments.

Findings

Success bias amplifies the illusion of understanding.

Communities favoring successful theories explore fewer options.

Optimizing for perceived success can worsen actual performance.

Abstract

Success-driven social learning, in which individuals preferentially adopt the ideas and methods that appear most successful, is a foundational principle of collective behavior across systems ranging from ant colonies to scientific communities. But science is a particular kind of collective search -- one in which the quality of an explanation is itself difficult to assess. Is success bias adaptive in this setting? In agent-based simulations of collective theory building, we find that it is not. Scientists in our model systematically overestimate the quality of their own theories, creating an illusion of understanding: a persistent gap between perceived and actual performance. Success bias amplifies this illusion; communities that favor apparently successful theories explore a narrower range of possibilities, efficiently filtering out poor explanations but failing to discover better ones. This effect intensifies with problem complexity, as scientists in more complex environments become increasingly unable to assess how well their theories actually perform. Most strikingly, when agents optimize their social behavior to maximize the perceived success of their theories, they paradoxically undermine their actual performance, and produce levels of inequality that mirror those found in real scientific communities.