Efficacy of BERT embeddings on predicting disaster from Twitter data

TL;DR

This study evaluates the effectiveness of BERT contextual embeddings in predicting disasters from Twitter data, demonstrating superior performance over traditional embeddings using various machine learning models.

Contribution

It provides a comprehensive analysis of BERT embeddings' impact on disaster tweet classification, comparing them with traditional embeddings and offering both quantitative and qualitative insights.

Findings

BERT embeddings outperform traditional embeddings in disaster prediction.

Deep learning models achieve higher accuracy with BERT.

The study offers accessible code for further research.

Abstract

Social media like Twitter provide a common platform to share and communicate personal experiences with other people. People often post their life experiences, local news, and events on social media to inform others. Many rescue agencies monitor this type of data regularly to identify disasters and reduce the risk of lives. However, it is impossible for humans to manually check the mass amount of data and identify disasters in real-time. For this purpose, many research works have been proposed to present words in machine-understandable representations and apply machine learning methods on the word representations to identify the sentiment of a text. The previous research methods provide a single representation or embedding of a word from a given document. However, the recent advanced contextual embedding method (BERT) constructs different vectors for the same word in different contexts. BERT embeddings have been successfully used in different natural language processing (NLP) tasks, yet there is no concrete analysis of how these representations are helpful in disaster-type tweet analysis. In this research work, we explore the efficacy of BERT embeddings on predicting disaster from Twitter data and compare these to traditional context-free word embedding methods (GloVe, Skip-gram, and FastText). We use both traditional machine learning methods and deep learning methods for this purpose. We provide both quantitative and qualitative results for this study. The results show that the BERT embeddings have the best results in disaster prediction task than the traditional word embeddings. Our codes are made freely accessible to the research community.