Finding Trolls Under Bridges: Preliminary Work on a Motif Detector

TL;DR

This paper introduces a preliminary system for automatically detecting motifs in texts, aiming to facilitate culturally-aware natural language processing by capturing recurring folkloric elements.

Contribution

It presents an initial architecture and an annotation effort for training motif detection, including testing a metaphor detector as a feature for identifying motifs.

Findings

Metaphor detector achieves F1 of 0.35 on motifs

Macro-average F1 of 0.21 across four motif categories

Annotation effort for motif data is ongoing

Abstract

Motifs are distinctive recurring elements found in folklore that have significance as communicative devices in news, literature, press releases, and propaganda. Motifs concisely imply a large constellation of culturally-relevant information, and their broad usage suggests their cognitive importance as touchstones of cultural knowledge, making their detection a worthy step toward culturally-aware natural language processing tasks. Until now, folklorists and others interested in motifs have only extracted motifs from narratives manually. We present a preliminary report on the development of a system for automatically detecting motifs. We briefly describe an annotation effort to produce data for training motif detection, which is on-going. We describe our in-progress architecture in detail, which aims to capture, in part, how people determine whether or not a motif candidate is being used in a motific way. This description includes a test of an off-the-shelf metaphor detector as a feature for motif detection, which achieves a F1 of 0.35 on motifs and a macro-average F1 of 0.21 across four categories which we assign to motif candidates.