Behind Maya: Building a Multilingual Vision Language Model

TL;DR

Maya is an open-source multilingual vision-language model trained on a diverse dataset in eight languages, aiming to improve performance on low-resource languages and cultural contexts in vision-language tasks.

Contribution

The paper introduces Maya, a novel multilingual VLM with a new dataset and model supporting eight languages, addressing limitations of existing models in low-resource and diverse cultural settings.

Findings

Enhanced performance on multilingual vision-language benchmarks

Effective cultural and linguistic comprehension in low-resource languages

Open-source code available for community use

Abstract

In recent times, we have seen a rapid development of large Vision-Language Models (VLMs). They have shown impressive results on academic benchmarks, primarily in widely spoken languages but lack performance on low-resource languages and varied cultural contexts. To address these limitations, we introduce Maya, an open-source Multilingual VLM. Our contributions are: 1) a multilingual image-text pretraining dataset in eight languages, based on the LLaVA pretraining dataset; and 2) a multilingual image-text model supporting these languages, enhancing cultural and linguistic comprehension in vision-language tasks. Code available at https://github.com/nahidalam/maya.