Multi-Query Focused Disaster Summarization via Instruction-Based Prompting
Philipp Seeberger, Korbinian Riedhammer

TL;DR
This paper presents a multi-stream disaster summarization method using retrieval, reranking, and instruction-based prompting with LLaMA-13b, demonstrating strong results in extracting key facts from social media and web sources during emergencies.
Contribution
It introduces a novel combination of retrieval, reranking, and instruction-following summarization with open-source LLMs for disaster event summarization.
Findings
Effective retrieval and reranking pipeline using BM25 and MonoT5
Question Answering prompting improves fact extraction accuracy
Open-source LLMs outperform baseline methods but lag behind proprietary systems
Abstract
Automatic summarization of mass-emergency events plays a critical role in disaster management. The second edition of CrisisFACTS aims to advance disaster summarization based on multi-stream fact-finding with a focus on web sources such as Twitter, Reddit, Facebook, and Webnews. Here, participants are asked to develop systems that can extract key facts from several disaster-related events, which ultimately serve as a summary. This paper describes our method to tackle this challenging task. We follow previous work and propose to use a combination of retrieval, reranking, and an embarrassingly simple instruction-following summarization. The two-stage retrieval pipeline relies on BM25 and MonoT5, while the summarizer module is based on the open-source Large Language Model (LLM) LLaMA-13b. For summarization, we explore a Question Answering (QA)-motivated prompting approach and find the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Semantic Web and Ontologies · Data Quality and Management
MethodsFocus
