Semantometrics: Towards Fulltext-based Research Evaluation

TL;DR

This paper explores semantometrics, a new approach to research evaluation that uses full-text semantic analysis to assess a publication's value, aiming to overcome limitations of citation-based metrics.

Contribution

It introduces and analyzes the semantometric contribution measure, comparing it with traditional citation-based metrics for research evaluation.

Findings

Semantic similarity can effectively estimate research contribution.

Semantometrics provide a complementary perspective to citation metrics.

The contribution measure shows promising correlations with research impact.

Abstract

Over the recent years, there has been a growing interest in developing new research evaluation methods that could go beyond the traditional citation-based metrics. This interest is motivated on one side by the wider availability or even emergence of new information evidencing research performance, such as article downloads, views and Twitter mentions, and on the other side by the continued frustrations and problems surrounding the application of purely citation-based metrics to evaluate research performance in practice. Semantometrics are a new class of research evaluation metrics which build on the premise that full-text is needed to assess the value of a publication. This paper reports on the analysis carried out with the aim to investigate the properties of the semantometric contribution measure, which uses semantic similarity of publications to estimate research contribution, and provides a comparative study of the contribution measure with traditional bibliometric measures based on citation counting.