Transformer and Hybrid Deep Learning Based Models for Machine-Generated Text Detection

TL;DR

This paper presents transformer and hybrid deep learning models for detecting machine-generated text, achieving high accuracy in a shared task but facing overfitting challenges that suggest areas for further improvement.

Contribution

It introduces transformer-based and hybrid models specifically designed for multi-generator, multi-domain, and multilingual text detection tasks.

Findings

Transformer model achieved 86.95% accuracy, second place in the competition.

Models showed overfitting issues, indicating need for regularization and tuning.

Hybrid model struggled with token-level transition detection due to overfitting.

Abstract

This paper describes the approach of the UniBuc - NLP team in tackling the SemEval 2024 Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection. We explored transformer-based and hybrid deep learning architectures. For subtask B, our transformer-based model achieved a strong \textbf{second-place} out of $77$ teams with an accuracy of \textbf{86.95\%}, demonstrating the architecture's suitability for this task. However, our models showed overfitting in subtask A which could potentially be fixed with less fine-tunning and increasing maximum sequence length. For subtask C (token-level classification), our hybrid model overfit during training, hindering its ability to detect transitions between human and machine-generated text.