ThreatCore: A Benchmark for Explicit and Implicit Threat Detection
Davide Bruni, Carlo Bardazzi, Maurizio Tesconi

TL;DR
ThreatCore introduces a comprehensive benchmark dataset for fine-grained threat detection in NLP, distinguishing explicit and implicit threats, and evaluates current models showing significant detection challenges especially for implicit threats.
Contribution
This work presents ThreatCore, a new benchmark dataset with re-annotated and augmented data, enabling more consistent and detailed threat detection research in NLP.
Findings
Implicit threats are harder to detect than explicit ones.
Semantic Role Labeling improves threat detection performance.
Current models struggle with identifying indirect harmful intent.
Abstract
Threat detection in Natural Language Processing lacks consistent definitions and standardized benchmarks, and is often conflated with broader phenomena such as toxicity, hate speech, or offensive language. In this work, we introduce ThreatCore, a public available benchmark dataset for fine-grained threat detection that distinguishes between explicit threats, implicit threats, and non-threats. The dataset is constructed by aggregating multiple publicly available resources and systematically re-annotating them under a unified operational definition of threat, revealing substantial inconsistencies across existing labels. To improve the coverage of underrepresented cases, particularly implicit threats, we further augment the dataset with synthetic examples, which are manually validated using the same annotation protocol adopted for the re-annotation of the public datasets, ensuring…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
