CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding

TL;DR

This paper introduces CrisisBERT, a transformer-based model for crisis classification and embedding, demonstrating improved accuracy, robustness, and novel application of attention-based deep neural networks in crisis detection from social media data.

Contribution

It presents the first use of transformer models for crisis classification and document-level crisis embedding, outperforming traditional methods and showing robustness with limited data.

Findings

CrisisBERT achieves higher accuracy and F1 scores in crisis detection.

The model maintains performance with 51.4% additional data across 6 to 36 events.

Crisis2Vec outperforms Word2Vec and GloVe in crisis embedding tasks.

Abstract

Classification of crisis events, such as natural disasters, terrorist attacks and pandemics, is a crucial task to create early signals and inform relevant parties for spontaneous actions to reduce overall damage. Despite crisis such as natural disasters can be predicted by professional institutions, certain events are first signaled by civilians, such as the recent COVID-19 pandemics. Social media platforms such as Twitter often exposes firsthand signals on such crises through high volume information exchange over half a billion tweets posted daily. Prior works proposed various crisis embeddings and classification using conventional Machine Learning and Neural Network models. However, none of the works perform crisis embedding and classification using state of the art attention-based deep neural networks models, such as Transformers and document-level contextual embeddings. This work proposes CrisisBERT, an end-to-end transformer-based model for two crisis classification tasks, namely crisis detection and crisis recognition, which shows promising results across accuracy and f1 scores. The proposed model also demonstrates superior robustness over benchmark, as it shows marginal performance compromise while extending from 6 to 36 events with only 51.4% additional data points. We also proposed Crisis2Vec, an attention-based, document-level contextual embedding architecture for crisis embedding, which achieve better performance than conventional crisis embedding methods such as Word2Vec and GloVe. To the best of our knowledge, our works are first to propose using transformer-based crisis classification and document-level contextual crisis embedding in the literature.