Decision-Focused Summarization

TL;DR

This paper introduces decision-focused summarization, a method that creates summaries tailored to support specific decisions by selecting relevant information that aligns with decision-making models, demonstrated on restaurant reviews.

Contribution

The paper proposes a novel decision-focused summarization approach that improves decision support by selecting non-redundant, decision-faithful summaries, outperforming existing methods.

Findings

DecSum outperforms text-only and explanation-based methods in decision faithfulness.

DecSum enables humans to better predict future restaurant ratings.

The approach effectively summarizes relevant information for decision support.

Abstract

Relevance in summarization is typically defined based on textual information alone, without incorporating insights about a particular decision. As a result, to support risk analysis of pancreatic cancer, summaries of medical notes may include irrelevant information such as a knee injury. We propose a novel problem, decision-focused summarization, where the goal is to summarize relevant information for a decision. We leverage a predictive model that makes the decision based on the full text to provide valuable insights on how a decision can be inferred from text. To build a summary, we then select representative sentences that lead to similar model decisions as using the full text while accounting for textual non-redundancy. To evaluate our method (DecSum), we build a testbed where the task is to summarize the first ten reviews of a restaurant in support of predicting its future rating on Yelp. DecSum substantially outperforms text-only summarization methods and model-based explanation methods in decision faithfulness and representativeness. We further demonstrate that DecSum is the only method that enables humans to outperform random chance in predicting which restaurant will be better rated in the future.