Revisiting Cross-Lingual Summarization: A Corpus-based Study and A New Benchmark with Improved Annotation

TL;DR

This paper introduces ConvSumX, a new cross-lingual summarization benchmark with improved annotation considering source context, and proposes a 2-Step method that outperforms existing baselines by leveraging both conversation and summary inputs.

Contribution

The paper presents ConvSumX, a novel benchmark for cross-lingual summarization with explicit source context annotation, and a 2-Step method that enhances summarization quality by mimicking human annotation.

Findings

ConvSumX is more faithful to input text than existing corpora.

The 2-Step method outperforms strong baselines in automatic and human evaluations.

Both source input and summary are vital for effective cross-lingual summarization.

Abstract

Most existing cross-lingual summarization (CLS) work constructs CLS corpora by simply and directly translating pre-annotated summaries from one language to another, which can contain errors from both summarization and translation processes. To address this issue, we propose ConvSumX, a cross-lingual conversation summarization benchmark, through a new annotation schema that explicitly considers source input context. ConvSumX consists of 2 sub-tasks under different real-world scenarios, with each covering 3 language directions. We conduct thorough analysis on ConvSumX and 3 widely-used manually annotated CLS corpora and empirically find that ConvSumX is more faithful towards input text. Additionally, based on the same intuition, we propose a 2-Step method, which takes both conversation and summary as input to simulate human annotation process. Experimental results show that 2-Step method surpasses strong baselines on ConvSumX under both automatic and human evaluation. Analysis shows that both source input text and summary are crucial for modeling cross-lingual summaries.