How Does India Cook Biryani?

TL;DR

This paper introduces a large-scale dataset of regional biryani videos, a multi-stage vision-language framework for fine-grained procedural analysis, and a comprehensive benchmark for evaluating multimodal reasoning in cultural culinary videos.

Contribution

It presents the first curated dataset of regional biryani videos, a novel multi-stage VLM-based framework for procedural segmentation and comparison, and a new QA benchmark for multimodal reasoning tasks.

Findings

Effective fine-grained video segmentation and alignment with transcripts.

Automated identification and explanation of regional procedural differences.

Benchmark results showing state-of-the-art models' performance on cultural video reasoning.

Abstract

Biryani, one of India's most celebrated dishes, exhibits remarkable regional diversity in its preparation, ingredients, and presentation. With the growing availability of online cooking videos, there is unprecedented potential to study such culinary variations using computational tools systematically. However, existing video understanding methods fail to capture the fine-grained, multimodal, and culturally grounded differences in procedural cooking videos. This work presents the first large-scale, curated dataset of biryani preparation videos, comprising 120 high-quality YouTube recordings across 12 distinct regional styles. We propose a multi-stage framework leveraging recent advances in vision-language models (VLMs) to segment videos into fine-grained procedural units and align them with audio transcripts and canonical recipe text. Building on these aligned representations, we introduce a video comparison pipeline that automatically identifies and explains procedural differences between regional variants. We construct a comprehensive question-answer (QA) benchmark spanning multiple reasoning levels to evaluate procedural understanding in VLMs. Our approach employs multiple VLMs in complementary roles, incorporates human-in-the-loop verification for high-precision tasks, and benchmarks several state-of-the-art models under zero-shot and fine-tuned settings. The resulting dataset, comparison methodology, and QA benchmark provide a new testbed for evaluating VLMs on structured, multimodal reasoning tasks and open new directions for computational analysis of cultural heritage through cooking videos. We release all data, code, and the project website at https://farzanashaju.github.io/how-does-india-cook-biryani/.