Long Text and Multi-Table Summarization: Dataset and Method

TL;DR

This paper introduces FINDSum, a large-scale dataset for summarizing long reports with both text and multiple tables, and proposes new methods and metrics to improve multi-modal summarization.

Contribution

The paper presents the first large-scale dataset for long text and multi-table summarization and introduces methods and evaluation metrics that jointly consider textual and tabular data.

Findings

Joint consideration of text and tables improves summary quality.

The proposed dataset enables training and evaluation of multi-modal summarization models.

Evaluation metrics highlight the importance of numerical information in summaries.

Abstract

Automatic document summarization aims to produce a concise summary covering the input document's salient information. Within a report document, the salient information can be scattered in the textual and non-textual content. However, existing document summarization datasets and methods usually focus on the text and filter out the non-textual content. Missing tabular data can limit produced summaries' informativeness, especially when summaries require covering quantitative descriptions of critical metrics in tables. Existing datasets and methods cannot meet the requirements of summarizing long text and multiple tables in each report. To deal with the scarcity of available data, we propose FINDSum, the first large-scale dataset for long text and multi-table summarization. Built on 21,125 annual reports from 3,794 companies, it has two subsets for summarizing each company's results of operations and liquidity. To summarize the long text and dozens of tables in each report, we present three types of summarization methods. Besides, we propose a set of evaluation metrics to assess the usage of numerical information in produced summaries. Dataset analyses and experimental results indicate the importance of jointly considering input textual and tabular data when summarizing report documents.