Combining Word and Character Vector Representation on Neural Machine Translation

TL;DR

This paper explores various methods of combining word and character vector representations in neural machine translation to improve translation quality, demonstrating significant BLEU score improvements over baseline models.

Contribution

It introduces and compares six configurations of NMT models that integrate word and character vectors using different neural network architectures and operations, highlighting the most effective combination.

Findings

Concatenation of word and character vectors yields the highest BLEU score improvement.

The best model achieved a BLEU score of 42.48, significantly higher than the baseline.

Combining word and character representations generally enhances translation performance.

Abstract

This paper describes combinations of word vector representation and character vector representation in English-Indonesian neural machine translation (NMT). Six configurations of NMT models were built with different input vector representations: word-based, combination of word and character representation using bidirectional LSTM(bi-LSTM), combination of word and character representation using CNN, combination of word and character representation by combining bi-LSTM and CNN by three different vector operations: addition, pointwise multiplication, and averaging. The experiment results showed that NMT models with concatenation of word and character representation obtained BLEU score higher than baseline model, ranging from 9.14 points to 11.65 points, for all models that combining both word and character representation, except the model that combining word and character representation using both bi-LSTM and CNN by addition operation. The highest BLEU score achieved was 42.48 compared to the 30.83 of the baseline model.