Summary Explorer: Visualizing the State of the Art in Text Summarization

TL;DR

Summary Explorer is a visualization tool that enables qualitative assessment of 55 state-of-the-art text summarization systems across multiple datasets, focusing on coverage, faithfulness, and position bias.

Contribution

It introduces a novel visualization tool for manual inspection and comparison of summarization outputs, enhancing debugging and evaluation processes.

Findings

Supports analysis of 55 summarization systems

Visualizes coverage, faithfulness, and position bias

Improves qualitative assessment of summarization quality

Abstract

This paper introduces Summary Explorer, a new tool to support the manual inspection of text summarization systems by compiling the outputs of 55~state-of-the-art single document summarization approaches on three benchmark datasets, and visually exploring them during a qualitative assessment. The underlying design of the tool considers three well-known summary quality criteria (coverage, faithfulness, and position bias), encapsulated in a guided assessment based on tailored visualizations. The tool complements existing approaches for locally debugging summarization models and improves upon them. The tool is available at https://tldr.webis.de/