Summary Workbench: Unifying Application and Evaluation of Text Summarization Models

TL;DR

Summary Workbench is a versatile tool that simplifies developing and evaluating text summarization models by integrating new models and measures as plugins, offering visual analysis and flexible deployment options.

Contribution

It introduces a Docker-based, extensible platform for unified development and evaluation of text summarization models with visual analysis capabilities.

Findings

Supports easy integration of new models and measures

Provides visual insights into model strengths and weaknesses

Enables deployment for private resources

Abstract

This paper presents Summary Workbench, a new tool for developing and evaluating text summarization models. New models and evaluation measures can be easily integrated as Docker-based plugins, allowing to examine the quality of their summaries against any input and to evaluate them using various evaluation measures. Visual analyses combining multiple measures provide insights into the models' strengths and weaknesses. The tool is hosted at \url{https://tldr.demo.webis.de} and also supports local deployment for private resources.