Efficacy of Utilizing Large Language Models to Detect Public Threat Posted Online

TL;DR

This study evaluates the effectiveness of large language models like GPT-4 and PaLM in automatically detecting online threats, showing high accuracy and cost-efficiency, which could enhance content moderation but require ethical considerations.

Contribution

It demonstrates that LLMs can accurately identify online threats, providing a scalable tool for content moderation with a focus on performance and cost-effectiveness.

Findings

GPT-4 achieved 97.9% non-threat and 100% threat accuracy.

All models showed strong statistical significance in threat detection.

PaLM API was identified as highly cost-efficient.

Abstract

This paper examines the efficacy of utilizing large language models (LLMs) to detect public threats posted online. Amid rising concerns over the spread of threatening rhetoric and advance notices of violence, automated content analysis techniques may aid in early identification and moderation. Custom data collection tools were developed to amass post titles from a popular Korean online community, comprising 500 non-threat examples and 20 threats. Various LLMs (GPT-3.5, GPT-4, PaLM) were prompted to classify individual posts as either "threat" or "safe." Statistical analysis found all models demonstrated strong accuracy, passing chi-square goodness of fit tests for both threat and non-threat identification. GPT-4 performed best overall with 97.9% non-threat and 100% threat accuracy. Affordability analysis also showed PaLM API pricing as highly cost-efficient. The findings indicate LLMs can effectively augment human content moderation at scale to help mitigate emerging online risks. However, biases, transparency, and ethical oversight remain vital considerations before real-world implementation.