Measuring field-normalized impact of papers on specific societal groups: An altmetrics study based on Mendeley data

TL;DR

This study develops a target-oriented, field-normalized impact indicator based on Mendeley data to measure societal impact of papers on specific groups like students or educators, enhancing impact assessment precision.

Contribution

It extends the MNRS indicator to target-specific impact measurement, enabling evaluation of papers' influence on particular societal sectors such as education.

Findings

Demonstrates the indicator's ability to assess impact on specific societal groups.

Shows how journals and institutions can be evaluated for impact on targeted sectors.

Provides empirical examples of impact measurement on educational and teaching sectors.

Abstract

Bibliometrics is successful in measuring impact, because the target is clearly defined: the publishing scientist who is still active and working. Thus, citations are a target-oriented metric which measures impact on science. In contrast, societal impact measurements based on altmetrics are as a rule intended to measure impact in a broad sense on all areas of society (e.g. science, culture, politics, and economics). This tendency is especially reflected in the efforts to design composite indicators (e.g. the Altmetric attention score). We deem appropriate that not only the impact measurement using citations is target-oriented (citations measure the impact of papers on scientists), but also the measurement of impact using altmetrics. Impact measurements only make sense, if the target group - the recipient of academic papers - is clearly defined. Thus, we extend in this study the field-normalized reader impact indicator proposed by us in an earlier study, which is based on Mendeley data (the mean normalized reader score, MNRS), to a target-oriented field-normalized impact indicator (e.g., MNRS_ED measures reader impact on the sector of educational donation, i.e., teaching). This indicator can show - as demonstrated in empirical examples - the ability of journals, countries, and academic institutions to publish papers which are below or above the average impact of papers on a specific sector in society (e.g., the educational or teaching sector). For example, the method allows to measure the impact of scientific papers on students - controlling for the field in which the papers have been published and their publication year.