If there's a Trigger Warning, then where's the Trigger? Investigating Trigger Warnings at the Passage Level

TL;DR

This paper explores the challenge of identifying specific passages in texts that prompt trigger warnings, proposing methods to detect these passages both manually and automatically, and evaluating their effectiveness.

Contribution

It introduces a new dataset of annotated passages and evaluates the feasibility of automatic trigger passage detection using NLP classifiers.

Findings

Trigger annotation is subjective and challenging.

Automatic trigger classification is feasible but still difficult.

Fine-tuned and few-shot classifiers show promising results.

Abstract

Trigger warnings are labels that preface documents with sensitive content if this content could be perceived as harmful by certain groups of readers. Since warnings about a document intuitively need to be shown before reading it, authors usually assign trigger warnings at the document level. What parts of their writing prompted them to assign a warning, however, remains unclear. We investigate for the first time the feasibility of identifying the triggering passages of a document, both manually and computationally. We create a dataset of 4,135 English passages, each annotated with one of eight common trigger warnings. In a large-scale evaluation, we then systematically evaluate the effectiveness of fine-tuned and few-shot classifiers, and their generalizability. We find that trigger annotation belongs to the group of subjective annotation tasks in NLP, and that automatic trigger classification remains challenging but feasible.