Fast Training of NMT Model with Data Sorting

TL;DR

This paper introduces a data sorting algorithm for neural machine translation training that reduces computational waste by sorting sentence pairs by length, leading to faster training without sacrificing performance.

Contribution

The proposed partial sorting algorithm improves training efficiency in NMT models by minimizing unnecessary computation, applicable across different architectures.

Findings

Reduced training time in English-Korean and English-Luganda translation tasks

Maintained translation quality despite data sorting

Applicable to various Transformer-based models

Abstract

The Transformer model has revolutionized Natural Language Processing tasks such as Neural Machine Translation, and many efforts have been made to study the Transformer architecture, which increased its efficiency and accuracy. One potential area for improvement is to address the computation of empty tokens that the Transformer computes only to discard them later, leading to an unnecessary computational burden. To tackle this, we propose an algorithm that sorts translation sentence pairs based on their length before batching, minimizing the waste of computing power. Since the amount of sorting could violate the independent and identically distributed (i.i.d) data assumption, we sort the data partially. In experiments, we apply the proposed method to English-Korean and English-Luganda language pairs for machine translation and show that there are gains in computational time while maintaining the performance. Our method is independent of architectures, so that it can be easily integrated into any training process with flexible data lengths.