Revealing the impact of synthetic native samples and multi-tasking strategies in Hindi-English code-mixed humour and sarcasm detection

TL;DR

This study explores strategies like native sample mixing, multi-task learning, and large multilingual models to enhance Hindi-English code-mixed humour and sarcasm detection, finding that native samples and MTL significantly improve performance.

Contribution

The paper introduces the combined use of native sample mixing and multi-task learning for improved code-mixed humour and sarcasm detection, and evaluates the efficacy of large multilingual models.

Findings

Native sample mixing improves F1-score by up to 8.64%.

Multi-task learning boosts detection performance up to 12.35% in F1-score.

Prompting large multilingual models does not outperform native sample mixing or MTL.

Abstract

In this paper, we reported our experiments with various strategies to improve code-mixed humour and sarcasm detection. Particularly, we tried three approaches: (i) native sample mixing, (ii) multi-task learning (MTL), and (iii) prompting and instruction finetuning very large multilingual language models (VMLMs). In native sample mixing, we added monolingual task samples to code-mixed training sets. In MTL learning, we relied on native and code-mixed samples of a semantically related task (hate detection in our case). Finally, in our third approach, we evaluated the efficacy of VMLMs via few-shot context prompting and instruction finetuning. Some interesting findings we got are (i) adding native samples improved humor (raising the F1-score up to 6.76%) and sarcasm (raising the F1-score up to 8.64%) detection, (ii) training MLMs in an MTL framework boosted performance for both humour (raising the F1-score up to 10.67%) and sarcasm (increment up to 12.35% in F1-score) detection, and (iii) prompting and instruction finetuning VMLMs couldn't outperform the other approaches. Finally, our ablation studies and error analysis discovered the cases where our model is yet to improve. We provided our code for reproducibility.