Nutribullets Hybrid: Multi-document Health Summarization

TL;DR

This paper introduces a hybrid multi-document summarization method that effectively generates comparative health summaries by combining relation extraction, deterministic content aggregation, and language modeling, especially suited for domains with limited training data.

Contribution

The paper presents a novel hybrid approach for multi-document summarization that separates content selection from realization, enabling effective training with limited annotations in the health domain.

Findings

Produces more faithful and relevant summaries

Improves aggregation sensitivity in summaries

Maintains fluency comparable to traditional methods

Abstract

We present a method for generating comparative summaries that highlights similarities and contradictions in input documents. The key challenge in creating such summaries is the lack of large parallel training data required for training typical summarization systems. To this end, we introduce a hybrid generation approach inspired by traditional concept-to-text systems. To enable accurate comparison between different sources, the model first learns to extract pertinent relations from input documents. The content planning component uses deterministic operators to aggregate these relations after identifying a subset for inclusion into a summary. The surface realization component lexicalizes this information using a text-infilling language model. By separately modeling content selection and realization, we can effectively train them with limited annotations. We implemented and tested the model in the domain of nutrition and health -- rife with inconsistencies. Compared to conventional methods, our framework leads to more faithful, relevant and aggregation-sensitive summarization -- while being equally fluent.