Comparing Approaches to Automatic Summarization in Less-Resourced Languages

TL;DR

This paper compares various automatic summarization methods for less-resourced languages, highlighting the effectiveness of multilingual fine-tuned models over zero-shot prompting and translation-based approaches.

Contribution

It provides a comprehensive comparison of summarization techniques in low-resource languages, emphasizing the advantages of multilingual fine-tuning over other methods.

Findings

Multilingual fine-tuned mT5 outperforms most approaches in several metrics.

Zero-shot LLM prompting shows variable performance across models.

Translation pipeline approaches are less reliable for less-resourced languages.

Abstract

Automatic text summarization has achieved high performance in high-resourced languages like English, but comparatively less attention has been given to summarization in less-resourced languages. This work compares a variety of different approaches to summarization from zero-shot prompting of LLMs large and small to fine-tuning smaller models like mT5 with and without three data augmentation approaches and multilingual transfer. We also explore an LLM translation pipeline approach, translating from the source language to English, summarizing and translating back. Evaluating with five different metrics, we find that there is variation across LLMs in their performance across similar parameter sizes, that our multilingual fine-tuned mT5 baseline outperforms most other approaches including zero-shot LLM performance for most metrics, and that LLM as judge may be less reliable on less-resourced languages.