SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection

TL;DR

This paper reports on SemEval-2024 Task 8, which challenges systems to detect and analyze machine-generated text across multiple languages, domains, and tasks, highlighting the effectiveness of LLM-based approaches.

Contribution

It introduces a comprehensive multilingual, multidomain benchmark for machine-generated text detection with multiple subtasks and analyzes the performance of submitted systems.

Findings

LLMs are the most effective systems across all subtasks.

High participation indicates strong interest and relevance.

Performance varies across languages and tasks.

Abstract

We present the results and the main findings of SemEval-2024 Task 8: Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection. The task featured three subtasks. Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. This subtask has two tracks: a monolingual track focused solely on English texts and a multilingual track. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine. The task attracted a large number of participants: subtask A monolingual (126), subtask A multilingual (59), subtask B (70), and subtask C (30). In this paper, we present the task, analyze the results, and discuss the system submissions and the methods they used. For all subtasks, the best systems used LLMs.