From Classification to Generation: Insights into Crosslingual Retrieval Augmented ICL

TL;DR

This paper introduces CREA-ICL, a cross-lingual retrieval-augmented in-context learning method that enhances zero-shot classification performance of multilingual models by leveraging prompts from high-resource languages, though it faces challenges in generation tasks.

Contribution

The paper proposes a novel cross-lingual retrieval-augmented ICL approach that improves multilingual model performance in classification tasks by utilizing semantically similar prompts from high-resource languages.

Findings

Steady improvements in classification tasks with CREA-ICL

Challenges remain in applying CREA-ICL to generation tasks

Provides insights into performance dynamics of retrieval-augmented ICL

Abstract

The remarkable ability of Large Language Models (LLMs) to understand and follow instructions has sometimes been limited by their in-context learning (ICL) performance in low-resource languages. To address this, we introduce a novel approach that leverages cross-lingual retrieval-augmented in-context learning (CREA-ICL). By extracting semantically similar prompts from high-resource languages, we aim to improve the zero-shot performance of multilingual pre-trained language models (MPLMs) across diverse tasks. Though our approach yields steady improvements in classification tasks, it faces challenges in generation tasks. Our evaluation offers insights into the performance dynamics of retrieval-augmented in-context learning across both classification and generation domains.