Combining Deep Neural Reranking and Unsupervised Extraction for Multi-Query Focused Summarization

TL;DR

This paper presents a multi-component approach combining neural reranking, unsupervised extraction, and ILP/MMR frameworks to improve event-focused summarization in disaster scenarios, demonstrating strong results but also highlighting ongoing challenges.

Contribution

It introduces a novel combination of neural reranking, unsupervised extraction, and optimization frameworks for multi-query focused summarization in disaster event tracking.

Findings

Strong automatic scoring results across evaluation setups

Effective integration of neural and unsupervised methods

Identified challenges and shortcomings in current approaches

Abstract

The CrisisFACTS Track aims to tackle challenges such as multi-stream fact-finding in the domain of event tracking; participants' systems extract important facts from several disaster-related events while incorporating the temporal order. We propose a combination of retrieval, reranking, and the well-known Integer Linear Programming (ILP) and Maximal Marginal Relevance (MMR) frameworks. In the former two modules, we explore various methods including an entity-based baseline, pre-trained and fine-tuned Question Answering systems, and ColBERT. We then use the latter module as an extractive summarization component by taking diversity and novelty criteria into account. The automatic scoring runs show strong results across the evaluation setups but also reveal shortcomings and challenges.