Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learning

TL;DR

This paper introduces Sequential Reptile, a method that aligns gradients across tasks in multilingual learning to improve knowledge transfer and reduce negative transfer, outperforming existing approaches.

Contribution

The paper proposes a simple, efficient gradient alignment technique using sequential sampling and Reptile updates to enhance multilingual and multi-task learning performance.

Findings

Significantly reduces negative transfer and catastrophic forgetting.

Outperforms relevant baselines on multi-task and zero-shot transfer tasks.

Improves inter-task gradient alignment for better knowledge sharing.

Abstract

Multilingual models jointly pretrained on multiple languages have achieved remarkable performance on various multilingual downstream tasks. Moreover, models finetuned on a single monolingual downstream task have shown to generalize to unseen languages. In this paper, we first show that it is crucial for those tasks to align gradients between them in order to maximize knowledge transfer while minimizing negative transfer. Despite its importance, the existing methods for gradient alignment either have a completely different purpose, ignore inter-task alignment, or aim to solve continual learning problems in rather inefficient ways. As a result of the misaligned gradients between tasks, the model suffers from severe negative transfer in the form of catastrophic forgetting of the knowledge acquired from the pretraining. To overcome the limitations, we propose a simple yet effective method that can efficiently align gradients between tasks. Specifically, we perform each inner-optimization by sequentially sampling batches from all the tasks, followed by a Reptile outer update. Thanks to the gradients aligned between tasks by our method, the model becomes less vulnerable to negative transfer and catastrophic forgetting. We extensively validate our method on various multi-task learning and zero-shot cross-lingual transfer tasks, where our method largely outperforms all the relevant baselines we consider.