LIUM-CVC Submissions for WMT17 Multimodal Translation Task

TL;DR

This paper presents LIUM-CVC's neural machine translation systems for WMT17, integrating visual features into translation models, achieving top rankings in English-German and English-French tasks.

Contribution

Introduces novel multimodal NMT architectures utilizing visual features, leading to improved translation performance over previous models.

Findings

Ranked first in En-De and En-Fr translation tasks

Effective integration of visual context enhances translation quality

Achieved top scores on METEOR and BLEU metrics

Abstract

This paper describes the monomodal and multimodal Neural Machine Translation systems developed by LIUM and CVC for WMT17 Shared Task on Multimodal Translation. We mainly explored two multimodal architectures where either global visual features or convolutional feature maps are integrated in order to benefit from visual context. Our final systems ranked first for both En-De and En-Fr language pairs according to the automatic evaluation metrics METEOR and BLEU.