Embed2Detect: Temporally Clustered Embedded Words for Event Detection in Social Media

TL;DR

Embed2Detect is a novel event detection method for social media that leverages word embeddings and hierarchical clustering to incorporate semantic features, outperforming existing approaches in sports and political datasets.

Contribution

The paper introduces Embed2Detect, a new approach combining word embeddings with hierarchical clustering for improved semantic-aware event detection in social media.

Findings

Achieved 27% higher F-measure on sports data

Achieved 29% higher F-measure on political data

Outperforms several state-of-the-art methods

Abstract

Social media is becoming a primary medium to discuss what is happening around the world. Therefore, the data generated by social media platforms contain rich information which describes the ongoing events. Further, the timeliness associated with these data is capable of facilitating immediate insights. However, considering the dynamic nature and high volume of data production in social media data streams, it is impractical to filter the events manually and therefore, automated event detection mechanisms are invaluable to the community. Apart from a few notable exceptions, most previous research on automated event detection have focused only on statistical and syntactical features in data and lacked the involvement of underlying semantics which are important for effective information retrieval from text since they represent the connections between words and their meanings. In this paper, we propose a novel method termed Embed2Detect for event detection in social media by combining the characteristics in word embeddings and hierarchical agglomerative clustering. The adoption of word embeddings gives Embed2Detect the capability to incorporate powerful semantical features into event detection and overcome a major limitation inherent in previous approaches. We experimented our method on two recent real social media data sets which represent the sports and political domain and also compared the results to several state-of-the-art methods. The obtained results show that Embed2Detect is capable of effective and efficient event detection and it outperforms the recent event detection methods. For the sports data set, Embed2Detect achieved 27% higher F-measure than the best-performed baseline and for the political data set, it was an increase of 29%.