Leveraging the Global Research Infrastructure to Characterize the Impact of National Science Foundation Research

TL;DR

This paper analyzes how global research infrastructure data sources can help the NSF assess research impact, revealing differences in data coverage and metadata completeness across systems and suggesting improvements for data capture.

Contribution

It provides a comparative analysis of three research data systems, highlighting gaps and proposing actions to improve metadata quality for impact assessment.

Findings

CHORUS includes more NSF awards and papers than PAR

Time influences metadata inclusion across sources

Data journeys vary, affecting metadata completeness

Abstract

The Global Research infrastructure (GRI) is made up of the repositories and organizations that provide persistent identifiers (PIDs) and metadata for many kinds of research objects and connect these objects to funders, research institutions, researchers, and one another using PIDs. The INFORMATE Project has combined three data sources to focus on understanding how the global research infrastructure might help the US National Science Foundation (NSF) and other federal agencies identify and characterize the impact of their support. In this paper we present INFORMATE observations of three data systems. The NSF Award database represents NSF funding while the NSF Public Access Repository (PAR) and CHORUS, as a proxy for the GRI, represent two different view of results of that funding. We compare the first at the level of awards and the second two at the level of published research articles. Our findings demonstrate that CHORUS datasets include significantly more NSF awards and more related papers than does PAR. Our findings also suggest that time plays a significant role in the inclusion of award metadata across the sources analyzed. Data in those sources travel very different journeys, each presenting different obstacles to metadata completeness and suggesting necessary actions on the parts of authors and publishers to ensure that publication and funding metadata are captured. We discuss these actions, as well as implications our findings have for emergent technologies such as artificial intelligence and natural language processing.