Explaining (Sarcastic) Utterances to Enhance Affect Understanding in Multimodal Dialogues

TL;DR

This paper introduces MOSES, a neural network that explains sarcastic utterances in multimodal dialogues, improving affect understanding and downstream tasks like sarcasm detection, humor, and emotion recognition.

Contribution

It presents a novel deep learning model for generating explanations of sarcasm in dialogues, enhancing affective understanding and natural language understanding tasks.

Findings

MOSES outperforms state-of-the-art in sarcasm explanation by ~2%.

Generated explanations improve sarcasm detection by ~14% F1-score.

Leveraging explanations benefits humor and emotion recognition with ~2% F1-score improvements.

Abstract

Conversations emerge as the primary media for exchanging ideas and conceptions. From the listener's perspective, identifying various affective qualities, such as sarcasm, humour, and emotions, is paramount for comprehending the true connotation of the emitted utterance. However, one of the major hurdles faced in learning these affect dimensions is the presence of figurative language, viz. irony, metaphor, or sarcasm. We hypothesize that any detection system constituting the exhaustive and explicit presentation of the emitted utterance would improve the overall comprehension of the dialogue. To this end, we explore the task of Sarcasm Explanation in Dialogues, which aims to unfold the hidden irony behind sarcastic utterances. We propose MOSES, a deep neural network, which takes a multimodal (sarcastic) dialogue instance as an input and generates a natural language sentence as its explanation. Subsequently, we leverage the generated explanation for various natural language understanding tasks in a conversational dialogue setup, such as sarcasm detection, humour identification, and emotion recognition. Our evaluation shows that MOSES outperforms the state-of-the-art system for SED by an average of ~2% on different evaluation metrics, such as ROUGE, BLEU, and METEOR. Further, we observe that leveraging the generated explanation advances three downstream tasks for affect classification - an average improvement of ~14% F1-score in the sarcasm detection task and ~2% in the humour identification and emotion recognition task. We also perform extensive analyses to assess the quality of the results.