SemEval-2021 Task 6: Detection of Persuasion Techniques in Texts and Images

TL;DR

This paper presents SemEval-2021 task 6, which challenges systems to detect persuasion techniques in memes by analyzing texts and images, highlighting the importance of multimodal interaction modeling.

Contribution

It introduces a new dataset, annotation guidelines, and evaluation setup for detecting persuasion techniques in multimodal memes, and reports on the participating systems' approaches and results.

Findings

Multimodal analysis improves persuasion detection accuracy.

Joint modeling of text and image interactions benefits performance.

Teams exploring interaction modeling outperformed simple fusion methods.

Abstract

We describe SemEval-2021 task 6 on Detection of Persuasion Techniques in Texts and Images: the data, the annotation guidelines, the evaluation setup, the results, and the participating systems. The task focused on memes and had three subtasks: (i) detecting the techniques in the text, (ii) detecting the text spans where the techniques are used, and (iii) detecting techniques in the entire meme, i.e., both in the text and in the image. It was a popular task, attracting 71 registrations, and 22 teams that eventually made an official submission on the test set. The evaluation results for the third subtask confirmed the importance of both modalities, the text and the image. Moreover, some teams reported benefits when not just combining the two modalities, e.g., by using early or late fusion, but rather modeling the interaction between them in a joint model.