VietMEAgent: Culturally-Aware Few-Shot Multimodal Explanation for Vietnamese Visual Question Answering

TL;DR

VietMEAgent is a multimodal, explainable VQA system tailored for Vietnamese culture, integrating cultural knowledge and structured explanations to improve interpretability and cultural understanding in AI.

Contribution

The paper introduces VietMEAgent, a novel culturally-aware VQA framework with a dedicated Vietnamese cultural knowledge base and explainability modules, addressing cultural specificity and interpretability challenges.

Findings

Effective cultural object detection in Vietnamese images

Generation of transparent, human-readable explanations

Demonstrated on a new Vietnamese Cultural VQA dataset

Abstract

Contemporary Visual Question Answering (VQA) systems remain constrained when confronted with culturally specific content, largely because cultural knowledge is under-represented in training corpora and the reasoning process is not rendered interpretable to end users. This paper introduces VietMEAgent, a multimodal explainable framework engineered for Vietnamese cultural understanding. The method integrates a cultural object detection backbone with a structured program generation layer, yielding a pipeline in which answer prediction and explanation are tightly coupled. A curated knowledge base of Vietnamese cultural entities serves as an explicit source of background information, while a dual-modality explanation module combines attention-based visual evidence with structured, human-readable textual rationales. We further construct a Vietnamese Cultural VQA dataset sourced from public repositories and use it to demonstrate the practicality of programming-based methodologies for cultural AI. The resulting system provides transparent explanations that disclose both the computational rationale and the underlying cultural context, supporting education and cultural preservation with an emphasis on interpretability and cultural sensitivity.