Do Large Language Models Reduce Research Novelty? Evidence from Information Systems Journals

TL;DR

This study investigates whether the increased scholarly output driven by large language models like ChatGPT compromises research novelty, finding that authors in non-English countries show a significant decline in novelty, suggesting LLMs may influence research diversity.

Contribution

The paper provides empirical evidence on how LLMs impact research novelty, revealing a heterogeneous effect based on geographic and linguistic factors, and offers a theoretical interpretation through construal level theory.

Findings

Authors in non-English countries show a 0.18 SD decline in novelty.

The decline in novelty is robust across various specifications.

LLMs may shift researchers from abstract to concrete thinking.

Abstract

Large language models such as ChatGPT have increased scholarly output, but whether this productivity boost produces genuine intellectual advancement remains untested. I address this gap by measuring the semantic novelty of 13,847 articles published between 2020 and 2025 in 44 Information Systems journals. Using SPECTER2 embeddings, I operationalize novelty as the cosine distance between each paper and its nearest prior neighbors. A difference-in-differences design with the November 2022 release of ChatGPT as the treatment break reveals a heterogeneous pattern: authors affiliated with institutions in non-English-dominant countries show a 0.18 standard deviation decline in relative novelty compared to authors in English-dominant countries (beta = -0.176, p < 0.001), equivalent to a 7-percentile-point drop in the novelty distribution. This finding is robust across alternative novelty specifications, treatment break dates, and sub-samples, and survives a placebo test at a pre-treatment break. I interpret these results through the lens of construal level theory, proposing that LLMs function as proximity tools that shift researchers from abstract, exploratory thinking toward concrete, convention-following execution. The paper contributes to the growing debate on whether LLM-driven productivity gains come at the cost of intellectual diversity.