Where there's a will there's a way: ChatGPT is used more for science in countries where it is prohibited

TL;DR

This study shows that geographic restrictions on ChatGPT are ineffective in curbing its use in scientific research, as usage persists and even exceeds in restricted countries due to widespread workarounds.

Contribution

It provides empirical evidence that restricting AI services geographically does not significantly reduce usage, highlighting the need for alternative regulation strategies.

Findings

ChatGPT used in 12.6% of preprints by August 2023

Higher usage in restricted countries, 7.7% more than unrestricted

Restrictions did not decrease overall ChatGPT use in science

Abstract

Regulating AI is a key societal challenge, but which regulation methods are effective is unclear. This study measures the effectiveness of restricting AI services geographically, focusing on ChatGPT. OpenAI restricts ChatGPT access in several countries, including China and Russia. If restrictions are effective, ChatGPT use should be minimal in these countries. We measured use with a classifier based on distinctive word usage found in early versions of ChatGPT, e.g. "delve." We trained the classifier on pre- and post-ChatGPT "polished" abstracts and found it outperformed GPTZero and ZeroGPT on validation sets, including papers with self-reported AI use. Applying the classifier to preprints from Arxiv, BioRxiv, and MedRxiv showed ChatGPT was used in about 12.6% of preprints by August 2023, with 7.7% higher usage in restricted countries. The gap appeared before China's first major legal LLM became widely available. To test the possibility that, due to high demand, use in restricted countries would have been even higher without restrictions, we compared Asian countries with high expected demand (where English is not an official language) and found that use was higher in those with restrictions. ChatGPT use was correlated with higher views and downloads, but not citations or journal placement. Overall, restricting ChatGPT geographically has proven ineffective in science and possibly other domains, likely due to widespread workarounds.