Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators

TL;DR

This paper introduces feature-augmented transformer detectors that significantly improve robustness in AI-text detection across different domains and generators, outperforming previous models especially under distribution shifts.

Contribution

The study proposes a novel feature augmentation method combined with a DeBERTa backbone to enhance transferability and robustness of AI-text detectors across diverse datasets and generation methods.

Findings

Base models achieve up to 99.5% in-domain accuracy.

Feature augmentation improves transfer performance, reaching 85.9% on M4.

The proposed method outperforms zero-shot baselines by up to 7.22 points.

Abstract

AI-generated text is nowadays produced at scale across domains and heterogeneous generation pipelines, making robustness to distribution shift a central requirement for supervised binary detectors. We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation; this threshold is then kept fixed for all downstream test distributions, revealing domain- and generator-dependent error asymmetries under shift. We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile. Although base models achieve near-ceiling in-domain performance (up to 99.5% balanced accuracy), performance under shift is brittle and strongly model-dependent. Feature augmentation via attention-based linguistic feature fusion improves transfer, with our best model (DeBERTa-v3-base+FeatAttn) achieving 85.9% balanced accuracy on M4. Multi-seed experiments confirm high stability. Under the same fixed-threshold protocol, our model outperforms strong zero-shot baselines by up to +7.22 points. Category-level ablations further show that readability and vocabulary features contribute most to robustness under shift. Overall, these results demonstrate that feature augmentation and a modern DeBERTa backbone significantly outperform earlier BERT/RoBERTa models, while the fixed-threshold protocol provides a more realistic and informative assessment of practical detector robustness.