Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization

TL;DR

This paper empirically evaluates various multilingual tuning strategies for text summarization across 45 languages, establishing a new state-of-the-art and providing insights for future research.

Contribution

It provides a comprehensive comparison of three families of multilingual tuning strategies and introduces a new state-of-the-art on the XL-Sum dataset.

Findings

Established a new state-of-the-art on XL-Sum dataset

Compared five models across 45 languages

Provided insights for designing multilingual tuning strategies

Abstract

Recently, a large number of tuning strategies have been proposed to adapt pre-trained language models to downstream tasks. In this paper, we perform an extensive empirical evaluation of various tuning strategies for multilingual learning, particularly in the context of text summarization. Specifically, we explore the relative advantages of three families of multilingual tuning strategies (a total of five models) and empirically evaluate them for summarization over 45 languages. Experimentally, we not only established a new state-of-the-art on the XL-Sum dataset but also derive a series of observations that hopefully can provide hints for future research on the design of multilingual tuning strategies.