CrisisBench: Benchmarking Crisis-related Social Media Datasets for Humanitarian Information Processing
Firoj Alam, Hassan Sajjad, Muhammad Imran, Ferda Ofli

TL;DR
CrisisBench consolidates multiple crisis-related social media datasets and provides benchmarks to facilitate the development of better models for humanitarian crisis informatics tasks.
Contribution
This work merges existing datasets and offers standardized benchmarks, enabling more effective comparison and advancement in crisis informatics research.
Findings
Benchmark datasets improve model comparison
Deep learning architectures evaluated on crisis data
Consolidated datasets support better model training
Abstract
Time-critical analysis of social media streams is important for humanitarian organizations for planing rapid response during disasters. The \textit{crisis informatics} research community has developed several techniques and systems for processing and classifying big crisis-related data posted on social media. However, due to the dispersed nature of the datasets used in the literature (e.g., for training models), it is not possible to compare the results and measure the progress made towards building better models for crisis informatics tasks. In this work, we attempt to bridge this gap by combining various existing crisis-related datasets. We consolidate eight human-annotated datasets and provide 166.1k and 141.5k tweets for \textit{informativeness} and \textit{humanitarian} classification tasks, respectively. We believe that the consolidated dataset will help train more sophisticated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPublic Relations and Crisis Communication · Disaster Management and Resilience · Sentiment Analysis and Opinion Mining
MethodsLinear Layer · fastText · Residual Connection · Attention Dropout · Linear Warmup With Linear Decay · Weight Decay · Refunds@Expedia|||How do I get a full refund from Expedia? · Dense Connections · Adam · WordPiece
