DependencyAI: Detecting AI Generated Text through Dependency Parsing

TL;DR

DependencyAI is an interpretable method that detects AI-generated text using dependency relations, showing strong performance across languages and models, and revealing syntactic features that distinguish human from machine writing.

Contribution

The paper introduces DependencyAI, a linguistically grounded, dependency relation-based approach for detecting AI-generated text, emphasizing interpretability and cross-domain robustness.

Findings

DependencyAI achieves competitive detection accuracy.

Dependency relations reveal key syntactic differences.

Model overprediction varies across unseen domains.

Abstract

As large language models (LLMs) become increasingly prevalent, reliable methods for detecting AI-generated text are critical for mitigating potential risks. We introduce DependencyAI, a simple and interpretable approach for detecting AI-generated text using only the labels of linguistic dependency relations. Our method achieves competitive performance across monolingual, multi-generator, and multilingual settings. To increase interpretability, we analyze feature importance to reveal syntactic structures that distinguish AI-generated from human-written text. We also observe a systematic overprediction of certain models on unseen domains, suggesting that generator-specific writing styles may affect cross-domain generalization. Overall, our results demonstrate that dependency relations alone provide a robust signal for AI-generated text detection, establishing DependencyAI as a strong linguistically grounded, interpretable, and non-neural network baseline.