IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometry

TL;DR

This paper presents ELECTRA-based models with stylometric features for detecting machine-generated academic essays in English and Arabic, achieving top-tier F1-scores and competitive rankings in a benchmark challenge.

Contribution

It introduces language-specific ELECTRA models combined with stylometry for improved detection of AI-generated essays in multiple languages.

Findings

Achieved 99.7% F1-score in English detection

Achieved 98.4% F1-score in Arabic detection

Ranked 2nd in English and 1st in Arabic subtasks

Abstract

Recent research has investigated the problem of detecting machine-generated essays for academic purposes. To address this challenge, this research utilizes pre-trained, transformer-based models fine-tuned on Arabic and English academic essays with stylometric features. Custom models based on ELECTRA for English and AraELECTRA for Arabic were trained and evaluated using a benchmark dataset. Proposed models achieved excellent results with an F1-score of 99.7%, ranking 2nd among of 26 teams in the English subtask, and 98.4%, finishing 1st out of 23 teams in the Arabic one.