Discrete and Soft Prompting for Multilingual Models

TL;DR

This paper demonstrates that discrete and soft prompting methods outperform traditional finetuning in multilingual natural language inference tasks, especially in low-resource and crosslingual settings.

Contribution

It extends the effectiveness of prompting techniques from English to multilingual models, showing superior performance over finetuning in crosslingual transfer and multilingual training.

Findings

Prompting methods outperform finetuning in multilingual NLI.

Discrete prompting achieves 38.79% accuracy with limited English data.

Prompting shows strong results across multiple languages.

Abstract

It has been shown for English that discrete and soft prompting perform strongly in few-shot learning with pretrained language models (PLMs). In this paper, we show that discrete and soft prompting perform better than finetuning in multilingual cases: Crosslingual transfer and in-language training of multilingual natural language inference. For example, with 48 English training examples, finetuning obtains 33.74% accuracy in crosslingual transfer, barely surpassing the majority baseline (33.33%). In contrast, discrete and soft prompting outperform finetuning, achieving 36.43% and 38.79%. We also demonstrate good performance of prompting with training data in multiple languages other than English.