StylOch at PAN: Gradient-Boosted Trees with Frequency-Based Stylometric Features

TL;DR

This paper presents a gradient-boosted trees classifier using frequency-based stylometric features from a large corpus of machine-generated texts, emphasizing explainability and computational efficiency.

Contribution

It introduces a modular stylometric pipeline combining spaCy features with gradient boosting, trained on over 500,000 texts for improved AI detection.

Findings

Effective non-neural classifier for AI detection

Utilizes extensive linguistic feature set

Achieves high accuracy with computational efficiency

Abstract

This submission to the binary AI detection task is based on a modular stylometric pipeline, where: public spaCy models are used for text preprocessing (including tokenisation, named entity recognition, dependency parsing, part-of-speech tagging, and morphology annotation) and extracting several thousand features (frequencies of n-grams of the above linguistic annotations); light-gradient boosting machines are used as the classifier. We collect a large corpus of more than 500 000 machine-generated texts for the classifier's training. We explore several parameter options to increase the classifier's capacity and take advantage of that training set. Our approach follows the non-neural, computationally inexpensive but explainable approach found effective previously.