VAANI: Capturing the language landscape for an inclusive digital India

TL;DR

Project VAANI has created a comprehensive, multi-modal dataset capturing India's linguistic diversity with over 31,270 hours of speech and 289K images across 112 languages, supporting inclusive AI development.

Contribution

It introduces a large-scale, multi-modal dataset representing India's diverse languages and dialects, with rigorous quality checks and open-source data for inclusive speech and multimodal AI research.

Findings

Collected 31,270 hours of speech data across 112 languages.

Open-sourced 289K images and 2,067 hours of transcribed speech.

First large-scale dataset representing many Indian languages.

Abstract

Project VAANI is an initiative to create an India-representative multi-modal dataset that comprehensively maps India's linguistic diversity, starting with 165 districts across the country in its first two phases. Speech data is collected through a carefully structured process that uses image-based prompts to encourage spontaneous responses. Images are captured through a separate process that encompasses a broad range of topics, gathered from both within and across districts. The collected data undergoes a rigorous multi-stage quality evaluation, including both automated and manual checks to ensure highest possible standards in audio quality and transcription accuracy. Following this thorough validation, we have open-sourced around 289K images, approximately 31,270 hours of audio recordings, and around 2,067 hours of transcribed speech, encompassing 112 languages from 165 districts from 31 States and Union territories. Notably, significant of these languages are being represented for the first time in a dataset of this scale, making the VAANI project a groundbreaking effort in preserving and promoting linguistic inclusivity. This data can be instrumental in building inclusive speech models for India, and in advancing research and development across speech, image, and multimodal applications.