LoRaLay: A Multilingual and Multimodal Dataset for Long Range and Layout-Aware Summarization

TL;DR

LoRaLay introduces a multilingual, multimodal dataset with visual and layout information for long-range summarization, enabling better modeling of complex documents across multiple languages.

Contribution

It provides new datasets with layout information in multiple languages and proposes combined layout-aware and long-range models achieving state-of-the-art results.

Findings

Layout information improves summarization quality.

Multilingual datasets facilitate cross-lingual research.

Combined models outperform existing approaches.

Abstract

Text Summarization is a popular task and an active area of research for the Natural Language Processing community. By definition, it requires to account for long input texts, a characteristic which poses computational challenges for neural models. Moreover, real-world documents come in a variety of complex, visually-rich, layouts. This information is of great relevance, whether to highlight salient content or to encode long-range interactions between textual passages. Yet, all publicly available summarization datasets only provide plain text content. To facilitate research on how to exploit visual/layout information to better capture long-range dependencies in summarization models, we present LoRaLay, a collection of datasets for long-range summarization with accompanying visual/layout information. We extend existing and popular English datasets (arXiv and PubMed) with layout information and propose four novel datasets -- consistently built from scholar resources -- covering French, Spanish, Portuguese, and Korean languages. Further, we propose new baselines merging layout-aware and long-range models -- two orthogonal approaches -- and obtain state-of-the-art results, showing the importance of combining both lines of research.