Beyond checkmate: exploring the creative chokepoints in AI text

TL;DR

This study explores the nuanced differences between human and AI-generated texts across different segments, revealing key patterns that can improve detection methods and understanding of AI's creative limitations.

Contribution

It introduces a segment-specific analysis inspired by chess structure to identify distinctive features between human and AI texts, enhancing detection strategies.

Findings

AI texts resemble human writing in the body segment due to length

Deeper linguistic flow analysis shows higher divergence in AI and human texts

Human texts exhibit greater stylistic variation across segments

Abstract

The rapid advancement of Large Language Models (LLMs) has revolutionized text generation but also raised concerns about potential misuse, making detecting LLM-generated text (AI text) increasingly essential. While prior work has focused on identifying AI text and effectively checkmating it, our study investigates a less-explored territory: portraying the nuanced distinctions between human and AI texts across text segments (introduction, body, and conclusion). Whether LLMs excel or falter in incorporating linguistic ingenuity across text segments, the results will critically inform their viability and boundaries as effective creative assistants to humans. Through an analogy with the structure of chess games, comprising opening, middle, and end games, we analyze segment-specific patterns to reveal where the most striking differences lie. Although AI texts closely resemble human writing in the body segment due to its length, deeper analysis shows a higher divergence in features dependent on the continuous flow of language, making it the most informative segment for detection. Additionally, human texts exhibit greater stylistic variation across segments, offering a new lens for distinguishing them from AI. Overall, our findings provide fresh insights into human-AI text differences and pave the way for more effective and interpretable detection strategies. Codes available at https://github.com/tripto03/chess_inspired_human_ai_text_distinction.