Neural language representations predict outcomes of scientific research

TL;DR

This paper demonstrates that neural language models can predict scientific correlation outcomes from textual descriptions, aiding research planning and resource allocation in social sciences.

Contribution

It introduces a neural network model trained on a large dataset of social science findings to accurately predict correlations from text descriptions.

Findings

Neural network achieves high accuracy in predicting correlations.

Text descriptions contain sufficient information for reliable predictions.

Potential to guide future research and resource prioritization.

Abstract

Many research fields codify their findings in standard formats, often by reporting correlations between quantities of interest. But the space of all testable correlates is far larger than scientific resources can currently address, so the ability to accurately predict correlations would be useful to plan research and allocate resources. Using a dataset of approximately 170,000 correlational findings extracted from leading social science journals, we show that a trained neural network can accurately predict the reported correlations using only the text descriptions of the correlates. Accurate predictive models such as these can guide scientists towards promising untested correlates, better quantify the information gained from new findings, and has implications for moving artificial intelligence systems from predicting structures to predicting relationships in the real world.