How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data

TL;DR

This study investigates the prevalence of Web of Science papers mentioned in policy documents using Altmetric data, revealing less than 0.5% mention rate and emphasizing the need for ongoing tracking for impact assessment.

Contribution

It provides an empirical analysis of policy-related mentions of scientific papers using new altmetrics data from Altmetric, highlighting current limitations and future tracking needs.

Findings

Less than 0.5% of WoS papers are mentioned in policy documents

Mentions are sparse and vary across subject categories

Regular updates are needed for impact measurement

Abstract

In this short communication, we provide an overview of a relatively newly provided source of altmetrics data which could possibly be used for societal impact measurements in scientometrics. Recently, Altmetric - a start-up providing publication level metrics - started to make data for publications available which have been mentioned in policy-related documents. Using data from Altmetric, we study how many papers indexed in the Web of Science (WoS) are mentioned in policy-related documents. We find that less than 0.5% of the papers published in different subject categories are mentioned at least once in policy-related documents. Based on our results, we recommend that the analysis of (WoS) publications with at least one policy-related mention is repeated regularly (annually). Mentions in policy-related documents should not be used for impact measurement until new policy-related sites are tracked.