Cross-lingual Transfer of Abstractive Summarizer to Less-resource Language

TL;DR

This paper explores cross-lingual transfer for abstractive summarization, adapting an English model to Slovene by integrating a language model for target language evaluation, achieving comparable quality to target-only models.

Contribution

It introduces a novel approach to cross-lingual abstractive summarization by addressing decoder limitations with an additional language model, enabling effective transfer to less-resource languages.

Findings

Best model produces summaries similar in quality to target-only models

Automatic and human evaluations confirm high accuracy and acceptable readability

Models occasionally generate misleading or absurd content

Abstract

Automatic text summarization extracts important information from texts and presents the information in the form of a summary. Abstractive summarization approaches progressed significantly by switching to deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, a cross-lingual model transfer is successfully applied in less-resource languages. For summarization, the cross-lingual model transfer was not attempted due to a non-reusable decoder side of neural models that cannot correct target language generation. In our work, we use a pre-trained English summarization model based on deep neural networks and sequence-to-sequence architecture to summarize Slovene news articles. We address the problem of inadequate decoder by using an additional language model for the evaluation of the generated text in target language. We test several cross-lingual summarization models with different amounts of target data for fine-tuning. We assess the models with automatic evaluation measures and conduct a small-scale human evaluation. Automatic evaluation shows that the summaries of our best cross-lingual model are useful and of quality similar to the model trained only in the target language. Human evaluation shows that our best model generates summaries with high accuracy and acceptable readability. However, similar to other abstractive models, our models are not perfect and may occasionally produce misleading or absurd content.