Grand challenges in altmetrics: heterogeneity, data quality and dependencies

TL;DR

This paper discusses the main challenges in altmetrics, including heterogeneity, data quality issues, and dependencies on technology and data sources, which hinder its effectiveness in research evaluation.

Contribution

It provides a detailed analysis of current challenges in altmetrics, emphasizing the impact of heterogeneity, data quality, and dependencies, with insights from bibliometrics history.

Findings

Heterogeneity complicates defining a unified altmetrics framework.

Data quality issues affect accuracy and reproducibility.

Dependencies on APIs, DOIs, and platforms influence altmetrics reliability.

Abstract

As uptake among researchers is constantly increasing, social media are finding their way into scholarly communication and, under the umbrella term altmetrics, were introduced to research evaluation. Fueled by technological possibilities and an increasing demand to demonstrate impact beyond the scientific community, altmetrics received great attention as potential democratizers of the scientific reward system and indicators of societal impact. This paper focuses on current challenges of altmetrics. Heterogeneity, data quality and particular dependencies are identified as the three major issues and discussed in detail with a particular emphasis on past developments in bibliometrics. The heterogeneity of altmetrics mirrors the diversity of the types of underlying acts, most of which take place on social media platforms. This heterogeneity has made it difficult to establish a common definition or conceptual framework. Data quality issues become apparent in the lack of accuracy, consistency and replicability of various altmetrics, which is largely affected by the dynamic nature of social media events. It is further highlighted that altmetrics are shaped by technical possibilities and depend particularly on the availability of APIs and DOIs, are strongly dependent on data providers and aggregators, and potentially influenced by technical affordances of underlying platforms.