ChatGPT, is this real? The influence of generative AI on writing style in top-tier cybersecurity papers

TL;DR

This study examines how generative AI, especially ChatGPT, influences writing styles in top-tier cybersecurity research papers, revealing increased lexical complexity and marker-word usage post-2022.

Contribution

It provides the first large-scale analysis of stylistic changes in cybersecurity papers due to generative AI, highlighting increased complexity and marker-word trends after 2022.

Findings

Post-2022 increase in marker-word usage across venues

Gradual rise in lexical complexity over time

Emerging trend towards more complex language in cybersecurity papers

Abstract

With the release of ChatGPT in 2022, generative AI has significantly lowered the cost of polishing and rewriting text. Due to its widespread usage, conference organizers instated specific requirements researchers need to adhere to when using GenAI. When asked to rewrite text, GenAI can introduce stylistic changes, often concentrated to a handful of ``marker words`` commonly associated with AI usage. Prior large-scale studies in preprints and biomedical science report post-2022 discontinuities of those marker words and broad linguistic features. This paper investigates whether similar patterns appear in top-tier cybersecurity conference papers (NDSS, USENIX Security, IEEE S\&P, and ACM CCS) over the period 2000-2025. Using text extracted from paper PDFs, we compute lexical and syntactic metrics and track curated marker-word usage. Our findings reveal a gradual long-run drift toward higher lexical complexity and a pronounced post-2022 increase in marker-word usage across all venues showing an emerging trend towards more complex language in cybersecurity papers possibly hindering accessibility.