Evaluation of Abstractive Summarisation Models with Machine Translation in Deliberative Processes

TL;DR

This paper evaluates abstractive summarisation models combined with machine translation for summarising non-English deliberative texts, addressing language and narrative complexity challenges.

Contribution

It introduces a novel evaluation approach for summarising complex deliberative texts in multiple languages using translation-based methods.

Findings

Summaries are fluent, consistent, and relevant.

Translation-based summarisation is effective across languages.

Method is easily adaptable to various languages.

Abstract

We present work on summarising deliberative processes for non-English languages. Unlike commonly studied datasets, such as news articles, this deliberation dataset reflects difficulties of combining multiple narratives, mostly of poor grammatical quality, in a single text. We report an extensive evaluation of a wide range of abstractive summarisation models in combination with an off-the-shelf machine translation model. Texts are translated into English, summarised, and translated back to the original language. We obtain promising results regarding the fluency, consistency and relevance of the summaries produced. Our approach is easy to implement for many languages for production purposes by simply changing the translation model.