Quantitative Methods in Research Evaluation Citation Indicators, Altmetrics, and Artificial Intelligence
Mike Thelwall

TL;DR
This work critically examines the effectiveness of citation metrics, altmetrics, and AI in research evaluation, highlighting their strengths, limitations, and potential applications across various academic fields.
Contribution
It provides a comprehensive analysis of existing indicators and discusses the potential and limitations of AI and large language models in research assessment.
Findings
Citation data can be informative in some fields but are not definitive quality measures.
Altmetrics offer supplementary insights but have limitations in accuracy.
AI, including large language models, may assist in specific research evaluation tasks.
Abstract
This book critically analyses the value of citation data, altmetrics, and artificial intelligence to support the research evaluation of articles, scholars, departments, universities, countries, and funders. It introduces and discusses indicators that can support research evaluation and analyses their strengths and weaknesses as well as the generic strengths and weaknesses of the use of indicators for research assessment. The book includes evidence of the comparative value of citations and altmetrics in all broad academic fields primarily through comparisons against article level human expert judgements from the UK Research Excellence Framework 2021. It also discusses the potential applications of traditional artificial intelligence and large language models for research evaluation, with large scale evidence for the former. The book concludes that citation data can be informative and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImpact of AI and Big Data on Business and Society
