Large Language Model Enhanced Clustering for News Event Detection

TL;DR

This paper introduces a novel framework combining Large Language Models and clustering analysis to improve news event detection and categorization, with a new stability index to evaluate clustering robustness.

Contribution

The paper presents a new event detection framework that integrates LLM-based embeddings, pre- and post-event tasks, and a novel Cluster Stability Assessment Index (CSAI) for more accurate and robust news event clustering.

Findings

LLM embeddings significantly improve clustering robustness

Post-event tasks provide meaningful event summaries

CSAI effectively measures clustering validity

Abstract

The news landscape is continuously evolving, with an ever-increasing volume of information from around the world. Automated event detection within this vast data repository is essential for monitoring, identifying, and categorizing significant news occurrences across diverse platforms. This paper presents an event detection framework that leverages Large Language Models (LLMs) combined with clustering analysis to detect news events from the Global Database of Events, Language, and Tone (GDELT). The framework enhances event clustering through both pre-event detection tasks (keyword extraction and text embedding) and post-event detection tasks (event summarization and topic labelling). We also evaluate the impact of various textual embeddings on the quality of clustering outcomes, ensuring robust news categorization. Additionally, we introduce a novel Cluster Stability Assessment Index (CSAI) to assess the validity and robustness of clustering results. CSAI utilizes multiple feature vectors to provide a new way of measuring clustering quality. Our experiments indicate that the use of LLM embedding in the event detection framework has significantly improved the results, demonstrating greater robustness in terms of CSAI scores. Moreover, post-event detection tasks generate meaningful insights, facilitating effective interpretation of event clustering results. Overall, our experimental results indicate that the proposed framework offers valuable insights and could enhance the accuracy in news analysis and reporting.