GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification

TL;DR

This paper describes fine-tuning a multilingual encoder for Arabic AI-generated text detection, finding simple mean pooling outperforms complex methods, with human texts generally longer than machine-generated ones.

Contribution

The study evaluates pooling strategies for Arabic text classification, revealing simple mean pooling is most effective and highlighting data length differences between human and machine texts.

Findings

Mean pooling achieved an F1 of 0.75.

Complex pooling methods did not outperform mean pooling.

Human texts are significantly longer than machine-generated texts.

Abstract

We present our approach to the AbjadGenEval shared task on detecting AI-generated Arabic text. We fine-tuned the multilingual E5-large encoder for binary classification, and we explored several pooling strategies to pool token representations, including weighted layer pooling, multi-head attention pooling, and gated fusion. Interestingly, none of these outperformed simple mean pooling, which achieved an F1 of 0.75 on the test set. We believe this is because complex pooling methods introduce additional parameters that need more data to train properly, whereas mean pooling offers a stable baseline that generalizes well even with limited examples. We also observe a clear pattern in the data: human-written texts tend to be significantly longer than machine-generated ones.