Multi30K: Multilingual English-German Image Descriptions

TL;DR

The paper introduces the Multi30K dataset, a multilingual image description dataset with English and German annotations, aimed at advancing research in multilingual multimodal understanding.

Contribution

It provides a new dataset with professionally translated and crowdsourced German descriptions for the Flickr30K images, enabling multilingual multimodal research.

Findings

Dataset includes German translations for Flickr30K images.

Supports multilingual image description and multimodal machine translation.

Facilitates broader research in multilingual multimodal tasks.

Abstract

We introduce the Multi30K dataset to stimulate multilingual multimodal research. Recent advances in image description have been demonstrated on English-language datasets almost exclusively, but image description should not be limited to English. This dataset extends the Flickr30K dataset with i) German translations created by professional translators over a subset of the English descriptions, and ii) descriptions crowdsourced independently of the original English descriptions. We outline how the data can be used for multilingual image description and multimodal machine translation, but we anticipate the data will be useful for a broader range of tasks.