Make Satire Boring Again: Reducing Stylistic Bias of Satirical Corpus by Utilizing Generative LLMs

TL;DR

This paper introduces a debiasing method using generative LLMs to improve satire detection models' robustness and generalizability across domains and languages, addressing stylistic bias in satirical corpora.

Contribution

It proposes a novel debiasing approach leveraging generative large language models and introduces the Turkish Satirical News Dataset with detailed annotations.

Findings

Debiasing improves model robustness in cross-domain and cross-lingual satire detection.

The approach enhances generalizability for irony detection in Turkish and English.

Limited impact observed on causal language models like Llama-3.1.

Abstract

Satire detection is essential for accurately extracting opinions from textual data and combating misinformation online. However, the lack of diverse corpora for satire leads to the problem of stylistic bias which impacts the models' detection performances. This study proposes a debiasing approach for satire detection, focusing on reducing biases in training data by utilizing generative large language models. The approach is evaluated in both cross-domain (irony detection) and cross-lingual (English) settings. Results show that the debiasing method enhances the robustness and generalizability of the models for satire and irony detection tasks in Turkish and English. However, its impact on causal language models, such as Llama-3.1, is limited. Additionally, this work curates and presents the Turkish Satirical News Dataset with detailed human annotations, with case studies on classification, debiasing, and explainability.