FreSaDa: A French Satire Data Set for Cross-Domain Satire Detection

TL;DR

This paper introduces FreSaDa, a French satire dataset designed for cross-domain satire detection, and evaluates baseline classification methods along with an unsupervised domain adaptation approach that improves performance.

Contribution

The paper presents a new French satire dataset with a cross-domain setup and proposes an unsupervised domain adaptation method that enhances satire detection accuracy.

Findings

Domain-specific features improve classification performance

Unsupervised domain adaptation significantly boosts results

Cross-source evaluation reveals challenges in satire detection

Abstract

In this paper, we introduce FreSaDa, a French Satire Data Set, which is composed of 11,570 articles from the news domain. In order to avoid reporting unreasonably high accuracy rates due to the learning of characteristics specific to publication sources, we divided our samples into training, validation and test, such that the training publication sources are distinct from the validation and test publication sources. This gives rise to a cross-domain (cross-source) satire detection task. We employ two classification methods as baselines for our new data set, one based on low-level features (character n-grams) and one based on high-level features (average of CamemBERT word embeddings). As an additional contribution, we present an unsupervised domain adaptation method based on regarding the pairwise similarities (given by the dot product) between the training samples and the validation samples as features. By including these domain-specific features, we attain significant improvements for both character n-grams and CamemBERT embeddings.