KLearn: Background Knowledge Inference from Summarization Data

TL;DR

This paper introduces a framework for inferring background knowledge from summarization data, enhancing summary scoring by explicitly modeling background information and improving alignment with human judgments.

Contribution

It develops techniques to infer background knowledge from summarization annotations and incorporates this into summary scoring functions, advancing understanding of human summarization behavior.

Findings

Background knowledge inference improves summary scoring accuracy.

Averaging multiple annotators' background knowledge enhances performance.

The framework offers insights into human information importance priors.

Abstract

The goal of text summarization is to compress documents to the relevant information while excluding background information already known to the receiver. So far, summarization researchers have given considerably more attention to relevance than to background knowledge. In contrast, this work puts background knowledge in the foreground. Building on the realization that the choices made by human summarizers and annotators contain implicit information about their background knowledge, we develop and compare techniques for inferring background knowledge from summarization data. Based on this framework, we define summary scoring functions that explicitly model background knowledge, and show that these scoring functions fit human judgments significantly better than baselines. We illustrate some of the many potential applications of our framework. First, we provide insights into human information importance priors. Second, we demonstrate that averaging the background knowledge of multiple, potentially biased annotators or corpora greatly improves summary-scoring performance. Finally, we discuss potential applications of our framework beyond summarization.