ASPECTNEWS: Aspect-Oriented Summarization of News Documents

TL;DR

This paper introduces AspectNews, a dataset for aspect-oriented news summarization, and evaluates methods for generating focused summaries that address specific subtopics without in-domain training.

Contribution

It presents a new dataset for aspect-oriented summarization and compares training schemes to improve focused summaries in news domains.

Findings

Synthetic training data improves aspect-focused summarization.

Keyword sensitivity affects summary quality.

Focused summaries outperform generic and keyword-based methods.

Abstract

Generic summaries try to cover an entire document and query-based summaries try to answer document-specific questions. But real users' needs often fall in between these extremes and correspond to aspects, high-level topics discussed among similar types of documents. In this paper, we collect a dataset of realistic aspect-oriented summaries, AspectNews, which covers different subtopics about articles in news sub-domains. We annotate data across two domains of articles, earthquakes and fraud investigations, where each article is annotated with two distinct summaries focusing on different aspects for each domain. A system producing a single generic summary cannot concisely satisfy both aspects. Our focus in evaluation is how well existing techniques can generalize to these domains without seeing in-domain training data, so we turn to techniques to construct synthetic training data that have been used in query-focused summarization work. We compare several training schemes that differ in how strongly keywords are used and how oracle summaries are extracted. Our evaluation shows that our final approach yields (a) focused summaries, better than those from a generic summarization system or from keyword matching; (b) a system sensitive to the choice of keywords.