Towards the relationship between AIGC in manuscript writing and author profiles: evidence from preprints in LLMs

TL;DR

This study investigates how AIGC tools like ChatGPT influence manuscript writing and author profiles, revealing usage patterns, correlations with author backgrounds, and implications for academic norms.

Contribution

It provides empirical evidence on AIGC adoption in scientific writing and its relationship with author characteristics and publication history.

Findings

Increase in AI-generated abstracts since ChatGPT release

Lower AIGC use among English-speaking scientists

Weak correlation between AI-generated probability and academic performance

Abstract

AIGC tools such as ChatGPT have profoundly changed scientific research, leading to widespread attention on its use on academic writing. Leveraging preprints from large language models, this study examined the use of AIGC in manuscript writing and its correlation with author profiles. We found that: (1) since the release of ChatGPT, the likelihood of abstracts being AI-generated has gradually increased; (2) scientists from English-speaking countries are less likely to use AIGC tools for writing assistance, while those from countries with linguistic differences from English are more likely to use these tools; (3) there is weak correlation between a paper's AI-generated probability and authors' academic performance; and (4) authors who have previously published papers with high AI-generated probabilities are more likely to continue using AIGC tools. We believe that this paper provides insightful results for relevant policies and norms and in enhancing the understanding of the relationship between humans and AI.