Multilingual Few-Shot Learning via Language Model Retrieval

TL;DR

This paper explores a retrieval-based approach for multilingual few-shot learning, demonstrating that selecting semantically similar samples improves model performance across various natural language understanding tasks in multiple languages.

Contribution

It introduces a retrieval method for selecting relevant few-shot samples, enhancing multilingual and cross-lingual model performance without gradient updates.

Findings

Consistently outperforms random sampling in monolingual tasks.

Improves accuracy in cross-lingual settings.

Effective across multiple NLP tasks.

Abstract

Transformer-based language models have achieved remarkable success in few-shot in-context learning and drawn a lot of research interest. However, these models' performance greatly depends on the choice of the example prompts and also has high variability depending on how samples are chosen. In this paper, we conduct a comprehensive study of retrieving semantically similar few-shot samples and using them as the context, as it helps the model decide the correct label without any gradient update in the multilingual and cross-lingual settings. We evaluate the proposed method on five natural language understanding datasets related to intent detection, question classification, sentiment analysis, and topic classification. The proposed method consistently outperforms random sampling in monolingual and cross-lingual tasks in non-English languages.