KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Cultures

TL;DR

Kahani is a culturally nuanced visual storytelling tool for non-Western cultures that leverages GPT-4 Turbo and SDXL to generate culturally grounded stories, outperforming general models in user evaluations.

Contribution

This paper introduces Kahani, a novel tool that generates culturally specific visual stories for non-Western cultures using advanced prompting techniques.

Findings

Kahani produces more culturally nuanced stories than ChatGPT-4 with DALL-E3.

In 75% of comparisons, Kahani outperformed or matched ChatGPT-4.

User study confirms Kahani's effectiveness in capturing cultural nuances.

Abstract

Large Language Models (LLMs) and Text-To-Image (T2I) models have demonstrated the ability to generate compelling text and visual stories. However, their outputs are predominantly aligned with the sensibilities of the Global North, often resulting in an outsider's gaze on other cultures. As a result, non-Western communities have to put extra effort into generating culturally specific stories. To address this challenge, we developed a visual storytelling tool called Kahani that generates culturally grounded visual stories for non-Western cultures. Our tool leverages off-the-shelf models GPT-4 Turbo and Stable Diffusion XL (SDXL). By using Chain of Thought (CoT) and T2I prompting techniques, we capture the cultural context from user's prompt and generate vivid descriptions of the characters and scene compositions. To evaluate the effectiveness of Kahani, we conducted a comparative user study with ChatGPT-4 (with DALL-E3) in which participants from different regions of India compared the cultural relevance of stories generated by the two tools. The results of the qualitative and quantitative analysis performed in the user study show that Kahani's visual stories are more culturally nuanced than those generated by ChatGPT-4. In 27 out of 36 comparisons, Kahani outperformed or was on par with ChatGPT-4, effectively capturing cultural nuances and incorporating more Culturally Specific Items (CSI), validating its ability to generate culturally grounded visual stories.