With Good MT There is No Need For End-to-End: A Case for Translate-then-Summarize Cross-lingual Summarization

TL;DR

This paper demonstrates that a translate-then-summarize pipeline for cross-lingual summarization consistently outperforms end-to-end models across 39 languages, emphasizing the value of traditional pipeline approaches over recent end-to-end trends.

Contribution

The study provides a comprehensive comparison showing the superiority of translate-then-summarize pipelines over end-to-end systems in cross-lingual summarization for multiple languages.

Findings

Translate-then-summarize pipeline outperforms end-to-end models on 39 languages.

System performance correlates with BLEU scores, aiding feasibility assessment.

Traditional pipeline approaches are more effective than end-to-end designs in this task.

Abstract

Recent work has suggested that end-to-end system designs for cross-lingual summarization are competitive solutions that perform on par or even better than traditional pipelined designs. A closer look at the evidence reveals that this intuition is based on the results of only a handful of languages or using underpowered pipeline baselines. In this work, we compare these two paradigms for cross-lingual summarization on 39 source languages into English and show that a simple \textit{translate-then-summarize} pipeline design consistently outperforms even an end-to-end system with access to enormous amounts of parallel data. For languages where our pipeline model does not perform well, we show that system performance is highly correlated with publicly distributed BLEU scores, allowing practitioners to establish the feasibility of a language pair a priori. Contrary to recent publication trends, our result suggests that the combination of individual progress of monolingual summarization and translation tasks offers better performance than an end-to-end system, suggesting that end-to-end designs should be considered with care.