Navigating Text-to-Image Generative Bias across Indic Languages

TL;DR

This paper introduces the IndicTTI benchmark to evaluate and analyze biases and performance of text-to-image models across 30 Indic languages, highlighting disparities and areas for improvement in multilingual generative AI.

Contribution

It presents the first comprehensive benchmark for assessing TTI models' support and biases in Indic languages, comparing multiple models and APIs across diverse linguistic contexts.

Findings

Identifies significant biases and performance gaps in TTI models for Indic languages.

Provides detailed analysis of model support for 30 Indic languages.

Offers resources and tools for further research and improvement.

Abstract

This research investigates biases in text-to-image (TTI) models for the Indic languages widely spoken across India. It evaluates and compares the generative performance and cultural relevance of leading TTI models in these languages against their performance in English. Using the proposed IndicTTI benchmark, we comprehensively assess the performance of 30 Indic languages with two open-source diffusion models and two commercial generation APIs. The primary objective of this benchmark is to evaluate the support for Indic languages in these models and identify areas needing improvement. Given the linguistic diversity of 30 languages spoken by over 1.4 billion people, this benchmark aims to provide a detailed and insightful analysis of TTI models' effectiveness within the Indic linguistic landscape. The data and code for the IndicTTI benchmark can be accessed at https://iab-rubric.org/resources/other-databases/indictti.