QuakeBERT: Accurate Classification of Social Media Texts for Rapid Earthquake Impact Assessment

TL;DR

QuakeBERT is a domain-specific large language model designed to classify and filter social media texts rapidly, improving earthquake impact assessment accuracy and aiding disaster response efforts.

Contribution

This paper introduces the first domain-specific LLM for earthquake-related social media classification and an integrated method for impact assessment combining opinion, sentiment, and keyword analysis.

Findings

QuakeBERT improves macro F1 score by 27% with increased data diversity and volume.

QuakeBERT outperforms CNN and RNN models, achieving an F1 score of 84.33%.

The approach effectively detects noisy microblogs, enhancing post-disaster response.

Abstract

Social media aids disaster response but suffers from noise, hindering accurate impact assessment and decision making for resilient cities, which few studies considered. To address the problem, this study proposes the first domain-specific LLM model and an integrated method for rapid earthquake impact assessment. First, a few categories are introduced to classify and filter microblogs considering their relationship to the physical and social impacts of earthquakes, and a dataset comprising 7282 earthquake-related microblogs from twenty earthquakes in different locations is developed as well. Then, with a systematic analysis of various influential factors, QuakeBERT, a domain-specific large language model (LLM), is developed and fine-tuned for accurate classification and filtering of microblogs. Meanwhile, an integrated method integrating public opinion trend analysis, sentiment analysis, and keyword-based physical impact quantification is introduced to assess both the physical and social impacts of earthquakes based on social media texts. Experiments show that data diversity and data volume dominate the performance of QuakeBERT and increase the macro average F1 score by 27%, while the best classification model QuakeBERT outperforms the CNN- or RNN-based models by improving the macro average F1 score from 60.87% to 84.33%. Finally, the proposed approach is applied to assess two earthquakes with the same magnitude and focal depth. Results show that the proposed approach can effectively enhance the impact assessment process by accurate detection of noisy microblogs, which enables effective post-disaster emergency responses to create more resilient cities.