Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation

TL;DR

This paper introduces Epi-Curriculum, a novel episodic training and denoised curriculum learning approach that significantly improves low-resource domain adaptation in neural machine translation, enhancing robustness and adaptability across domains.

Contribution

The paper proposes a new episodic training framework combined with denoised curriculum learning specifically designed for low-resource domain adaptation in NMT, which is a novel integration in this context.

Findings

Epi-Curriculum improves robustness and adaptability in seen and unseen domains.

Episodic training enhances encoder and decoder robustness to domain shifts.

The approach outperforms baseline methods on English-German and English-Romanian translation tasks.

Abstract

Neural Machine Translation (NMT) models have become successful, but their performance remains poor when translating on new domains with a limited number of data. In this paper, we present a novel approach Epi-Curriculum to address low-resource domain adaptation (DA), which contains a new episodic training framework along with denoised curriculum learning. Our episodic training framework enhances the model's robustness to domain shift by episodically exposing the encoder/decoder to an inexperienced decoder/encoder. The denoised curriculum learning filters the noised data and further improves the model's adaptability by gradually guiding the learning process from easy to more difficult tasks. Experiments on English-German and English-Romanian translation show that: (i) Epi-Curriculum improves both model's robustness and adaptability in seen and unseen domains; (ii) Our episodic training framework enhances the encoder and decoder's robustness to domain shift.