Cross-linguistic differences and similarities in image descriptions

TL;DR

This paper compares image descriptions across Dutch, English, and German, revealing similarities and the influence of worker familiarity on description detail.

Contribution

It provides the first cross-linguistic analysis of image descriptions, highlighting factors affecting description specificity across languages.

Findings

Descriptions are similar across languages in many aspects.

Worker familiarity impacts description specificity.

Cross-linguistic differences are influenced by cultural and contextual factors.

Abstract

Automatic image description systems are commonly trained and evaluated on large image description datasets. Recently, researchers have started to collect such datasets for languages other than English. An unexplored question is how different these datasets are from English and, if there are any differences, what causes them to differ. This paper provides a cross-linguistic comparison of Dutch, English, and German image descriptions. We find that these descriptions are similar in many respects, but the familiarity of crowd workers with the subjects of the images has a noticeable influence on description specificity.