HebID: Detecting Social Identities in Hebrew-language Political Text

TL;DR

HebID is a new multilabel Hebrew dataset for detecting nuanced social identities in political texts, enabling analysis of identity expression and differences between elite discourse and public priorities.

Contribution

It introduces the first Hebrew corpus for social identity detection, benchmarks models including LLMs, and applies analysis to political discourse and public survey data.

Findings

Hebrew-tuned LLMs achieve macro-F1 of 0.74

Identifies gender and temporal variations in identity expression

Reveals differences between elite discourse and public identity priorities

Abstract

Political language is deeply intertwined with social identities. While social identities are often shaped by specific cultural contexts and expressed through particular uses of language, existing datasets for group and identity detection are predominantly English-centric, single-label and focus on coarse identity categories. We introduce HebID, the first multilabel Hebrew corpus for social identity detection: 5,536 sentences from Israeli politicians' Facebook posts (Dec 2018-Apr 2021), manually annotated for twelve nuanced social identities (e.g. Rightist, Ultra-Orthodox, Socially-oriented) grounded by survey data. We benchmark multilabel and single-label encoders alongside 2B-9B-parameter generative LLMs, finding that Hebrew-tuned LLMs provide the best results (macro-$F_1$ = 0.74). We apply our classifier to politicians' Facebook posts and parliamentary speeches, evaluating differences in popularity, temporal trends, clustering patterns, and gender-related variations in identity expression. We utilize identity choices from a national public survey, enabling a comparison between identities portrayed in elite discourse and the public's identity priorities. HebID provides a comprehensive foundation for studying social identities in Hebrew and can serve as a model for similar research in other non-English political contexts.