Which stylistic features fool ChatGPT research evaluations?

TL;DR

This study investigates whether linguistic features like complexity and length influence ChatGPT's research quality scores, revealing potential biases towards more complex and lengthy abstracts that could affect research evaluation fairness.

Contribution

It provides empirical evidence that ChatGPT's scoring may be biased by language complexity and length, highlighting challenges in using LLMs for research assessment.

Findings

ChatGPT scores correlate more with linguistic complexity than with expert assessments.

Longer and less readable abstracts tend to receive higher ChatGPT scores.

Potential bias of LLMs towards complex language could impact research evaluation fairness.

Abstract

Large Language Models (LLMs) have the potential to be used to support research evaluation and have a moderate capability to estimate the research quality of a journal article from its title and abstract. This paper assesses whether there are language-related factors unrelated to the quality of the research that influence ChatGPT's scores. Using a dataset of 99,277 journal articles submitted to the UK-wide Research Excellence Framework (REF) 2021 assessments, we calculated several readability indicators from abstracts and correlated them with ChatGPT scores and departmental REF scores. From the results, linguistic complexity and length were more strongly associated with ChatGPT research quality scores than with REF expert scores in many subject areas. Although cause-and-effect was not tested, these results suggest that ChatGPT may be more likely than human experts to reward linguistic complexity, with a potential bias towards longer and less readable abstracts in many fields. The apparent preference of LLMs for complex language is an undesirable feature for practical applications of LLMs for research quality evaluation, unless solutions can be found.