Creating and Characterizing a Diverse Corpus of Sarcasm in Dialogue

TL;DR

This paper presents a large, diverse corpus of sarcasm in online debate dialogues, along with methods for identifying sarcasm types, enabling better understanding and detection of sarcasm in social media.

Contribution

It introduces a novel, high-quality sarcasm corpus with operationalized sarcasm classes and demonstrates improved sarcasm detection accuracy over prior work.

Findings

High-accuracy retrieval of sarcastic utterances using lexico-syntactic cues

Supervised learning achieves higher precision and F1 scores than previous methods

Linguistic analysis reveals distinct features of sarcasm classes

Abstract

The use of irony and sarcasm in social media allows us to study them at scale for the first time. However, their diversity has made it difficult to construct a high-quality corpus of sarcasm in dialogue. Here, we describe the process of creating a large- scale, highly-diverse corpus of online debate forums dialogue, and our novel methods for operationalizing classes of sarcasm in the form of rhetorical questions and hyperbole. We show that we can use lexico-syntactic cues to reliably retrieve sarcastic utterances with high accuracy. To demonstrate the properties and quality of our corpus, we conduct supervised learning experiments with simple features, and show that we achieve both higher precision and F than previous work on sarcasm in debate forums dialogue. We apply a weakly-supervised linguistic pattern learner and qualitatively analyze the linguistic differences in each class.