Impact of ChatGPT on the writing style of condensed matter physicists

TL;DR

This study uses a difference-in-differences approach to show that ChatGPT's release improved the English quality of abstracts by non-native speakers in condensed matter physics, indicating widespread adoption and style changes.

Contribution

It provides the first empirical evidence of ChatGPT's impact on scientific writing style in condensed matter physics using rigorous statistical analysis.

Findings

Improved English quality of abstracts by non-native speakers post-ChatGPT

Increase in unique words and decrease in rare words after ChatGPT release

Significant style changes for Latin and Ural-Altaic language authors

Abstract

We apply a state-of-the-art difference-in-differences approach to estimate the impact of ChatGPT's release on the writing style of condensed matter papers on arXiv. Our analysis reveals a statistically significant improvement in the English quality of abstracts written by non-native English speakers. Importantly, this improvement remains robust even after accounting for other potential factors, confirming that it can be attributed to the release of ChatGPT. This indicates widespread adoption of the tool. Following the release of ChatGPT, there is a significant increase in the use of unique words, while the frequency of rare words decreases. Across language families, the changes in writing style are significant for authors from the Latin and Ural-Altaic groups, but not for those from the Germanic or other Indo-European groups.