Generating Literature-Driven Scientific Theories at Scale

TL;DR

This paper introduces a scalable method for synthesizing scientific theories from literature, outperforming parametric models in evidence matching and future result prediction.

Contribution

It presents a novel approach to theory generation from literature, demonstrating improved accuracy and predictive capabilities over parametric knowledge methods.

Findings

Literature-supported theories better match existing evidence.

Theories predict future scientific results more accurately.

Scaling to 13.7k papers yields 2.9k theories.

Abstract

Contemporary automated scientific discovery has focused on agents for generating scientific experiments, while systems that perform higher-level scientific activities such as theory building remain underexplored. In this work, we formulate the problem of synthesizing theories consisting of qualitative and quantitative laws from large corpora of scientific literature. We study theory generation at scale, using 13.7k source papers to synthesize 2.9k theories, examining how generation using literature-grounding versus parametric knowledge, and accuracy-focused versus novelty-focused generation objectives change theory properties. Our experiments show that, compared to using parametric LLM memory for generation, our literature-supported method creates theories that are significantly better at both matching existing evidence and at predicting future results from 4.6k subsequently-written papers