Silent Signals, Loud Impact: LLMs for Word-Sense Disambiguation of Coded Dog Whistles

TL;DR

This paper introduces a novel LLM-based method for disambiguating coded dog whistles in speech, creating the largest dataset of such examples to aid hate speech detection and social science research.

Contribution

It presents a new approach using LLMs for word-sense disambiguation of dog whistles and releases the largest dataset of high-confidence coded examples.

Findings

Created a dataset of 16,550 disambiguated dog whistle examples

Demonstrated the effectiveness of LLMs in identifying coded language

Enabled applications in hate speech detection and political analysis

Abstract

A dog whistle is a form of coded communication that carries a secondary meaning to specific audiences and is often weaponized for racial and socioeconomic discrimination. Dog whistling historically originated from United States politics, but in recent years has taken root in social media as a means of evading hate speech detection systems and maintaining plausible deniability. In this paper, we present an approach for word-sense disambiguation of dog whistles from standard speech using Large Language Models (LLMs), and leverage this technique to create a dataset of 16,550 high-confidence coded examples of dog whistles used in formal and informal communication. Silent Signals is the largest dataset of disambiguated dog whistle usage, created for applications in hate speech detection, neology, and political science. The dataset can be found at https://huggingface.co/datasets/SALT-NLP/silent_signals.