IITP at WAT 2021: System description for English-Hindi Multimodal Translation Task

TL;DR

This paper describes a neural machine translation system for English-Hindi multimodal translation, leveraging visual information to improve translation quality in low-resource settings, and reports competitive BLEU scores on WAT 2021 datasets.

Contribution

It introduces a multimodal NMT approach for English-Hindi translation that utilizes visual context to enhance translation accuracy in low-resource scenarios.

Findings

Achieved BLEU scores of 42.47 and 37.50 on evaluation and challenge subsets.

Demonstrated the effectiveness of multimodal information in improving translation quality.

Participated in WAT 2021 with competitive results.

Abstract

Neural Machine Translation (NMT) is a predominant machine translation technology nowadays because of its end-to-end trainable flexibility. However, NMT still struggles to translate properly in low-resource settings specifically on distant language pairs. One way to overcome this is to use the information from other modalities if available. The idea is that despite differences in languages, both the source and target language speakers see the same thing and the visual representation of both the source and target is the same, which can positively assist the system. Multimodal information can help the NMT system to improve the translation by removing ambiguity on some phrases or words. We participate in the 8th Workshop on Asian Translation (WAT - 2021) for English-Hindi multimodal translation task and achieve 42.47 and 37.50 BLEU points for Evaluation and Challenge subset, respectively.