Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations

TL;DR

This paper investigates how deep contextualized representations can enhance determinantal point processes for extractive multi-document summarization, finding that combining deep features with surface indicators yields better summaries.

Contribution

It demonstrates the importance of integrating deep contextualized representations with surface features in DPP-based summarization models.

Findings

Deep representations alone are insufficient for optimal sentence selection.

Combining deep features with surface indicators improves summary quality.

Deep contextualized features complement traditional surface indicators in DPP models.

Abstract

Emerged as one of the best performing techniques for extractive summarization, determinantal point processes select the most probable set of sentences to form a summary according to a probability measure defined by modeling sentence prominence and pairwise repulsion. Traditionally, these aspects are modelled using shallow and linguistically informed features, but the rise of deep contextualized representations raises an interesting question of whether, and to what extent, contextualized representations can be used to improve DPP modeling. Our findings suggest that, despite the success of deep representations, it remains necessary to combine them with surface indicators for effective identification of summary sentences.