Measuring prominence of scientific work in online news as a proxy for impact

TL;DR

This paper proposes a novel method to measure the societal impact of scientific papers by analyzing their prominence in news articles, demonstrating that higher prominence correlates with higher impact scores.

Contribution

It introduces a new corpus linking news articles to scientific papers and a graph-based ranking algorithm to quantify their prominence as a proxy for societal impact.

Findings

Papers linked to high-impact REF case studies have higher prominence scores.

Prominence in news articles correlates with higher societal impact.

The proposed semantic similarity approach effectively measures news prominence.

Abstract

The impact made by a scientific paper on the work of other academics has many established metrics, including metrics based on citation counts and social media commenting. However, determination of the impact of a scientific paper on the wider society is less well established. For example, is it important for scientific work to be newsworthy? Here we present a new corpus of newspaper articles linked to the scientific papers that they describe. We find that Impact Case studies submitted to the UK Research Excellence Framework (REF) 2014 that refer to scientific papers mentioned in newspaper articles were awarded a higher score in the REF assessment. The papers associated with these case studies also feature prominently in the newspaper articles. We hypothesise that such prominence can be a useful proxy for societal impact. We therefore provide a novel baseline approach for measuring the prominence of scientific papers mentioned within news articles. Our measurement of prominence is based on semantic similarity through a graph-based ranking algorithm. We find that scientific papers with an associated REF case study are more likely to have a stronger prominence score. This supports our hypothesis that linguistic prominence in news can be used to suggest the wider non-academic impact of scientific work.