LM$^2$otifs : An Explainable Framework for Machine-Generated Texts Detection

TL;DR

LM$^2$otifs introduces an explainable graph neural network framework that effectively detects machine-generated texts while providing interpretable linguistic motifs, addressing the explainability gap in existing detection methods.

Contribution

It proposes a novel GNN-based framework with explainability for MGT detection, combining graph representations of text with motif extraction for interpretability.

Findings

Comparable detection performance to existing methods

Effective extraction of linguistically meaningful motifs

Distinct linguistic fingerprints identified in machine-generated texts

Abstract

The impressive ability of large language models to generate natural text across various tasks has led to critical challenges in authorship authentication. Although numerous detection methods have been developed to differentiate between machine-generated texts (MGT) and human-generated texts (HGT), the explainability of these methods remains a significant gap. Traditional explainability techniques often fall short in capturing the complex word relationships that distinguish HGT from MGT. To address this limitation, we present LM$^2$otifs, a novel explainable framework for MGT detection. Inspired by probabilistic graphical models, we provide a theoretical rationale for the effectiveness. LM$^2$otifs utilizes eXplainable Graph Neural Networks to achieve both accurate detection and interpretability. The LM$^2$otifs pipeline operates in three key stages: first, it transforms text into graphs based on word co-occurrence to represent lexical dependencies; second, graph neural networks are used for prediction; and third, a post-hoc explainability method extracts interpretable motifs, offering multi-level explanations from individual words to sentence structures. Extensive experiments on multiple benchmark datasets demonstrate the comparable performance of LM$^2$otifs. The empirical evaluation of the extracted explainable motifs confirms their effectiveness in differentiating HGT and MGT. Furthermore, qualitative analysis reveals distinct and visible linguistic fingerprints characteristic of MGT.