What indicators matter? The Analysis of Perception toward Research Assessment Indicators and Leiden Manifesto- The Case Study of Taiwan

TL;DR

This study investigates Taiwanese researchers' awareness of bibliometric indicators and the Leiden Manifesto, revealing limited recognition and understanding, and emphasizing the need for education to promote proper use in research evaluation.

Contribution

It provides empirical data on Taiwanese researchers' perceptions and knowledge of research indicators and principles, highlighting gaps and suggesting educational interventions.

Findings

Most researchers are only familiar with JIF and h-index.

There is limited understanding of the definition of relative citation ratio.

Researchers agree on the importance of field-normalization principles.

Abstract

This study aims to investigate the Taiwanese researchers' awareness toward bibliometric indicators and the principles from Leiden Manifesto. The online survey was conducted and obtained a total of 417 valid responses. The results show that evoking the right concept of use of bibliometric indicators and research evaluation has a long way to go. The lack of recognition of bibliometric indicators exists in Taiwanese academia. Generally speaking, researchers may hear of the certain indicator, but they are not familiar with its definition and calculation process. Only JIF and h-index are considered as well-known indicators. The results also suggest that the ten principles from Leiden Manifesto can be considered as the universal guideline in research evaluation since most of Taiwanese researchers agree the contents. Especially for the principle 6 "Account for variation by field in publication and citation practices" has highest degree of agreement. However, it is interesting to compare the result of recognition of relative citation ratio, only few researchers have fully understood the definition. This result indicates the scientometricians should make more effort to disseminate the concept of field-normalization in bibliometric indicators. The researchers do have understanding about the importance of comparison on the same basis, at the meantime, they may use the inappropriate indicators just because of lacking of enough knowledge on the variety of indicators. Hence, it is important to initiate the education of informetrics to all of the stakeholders in research evaluation so that the misuse and abuse of bibliometric indicators may possibly not happen again, and the bibliometric analysis is able to turn to contextualization-based analysis in the future.