Tourism Question Answer System in Indian Language using Domain-Adapted Foundation Models

TL;DR

This paper develops a Hindi tourism question-answering system focused on Varanasi, utilizing foundation models like BERT and RoBERTa with LoRA fine-tuning to achieve efficient and accurate answers in a low-resource, culturally nuanced domain.

Contribution

It introduces a new Hindi tourism QA dataset, evaluates multiple foundation models with LoRA fine-tuning, and demonstrates effective, resource-efficient performance for culturally specific applications.

Findings

LoRA fine-tuning achieves 85.3% F1 with 98% fewer parameters.

RoBERTa with SFT outperforms other models in capturing cultural nuances.

The dataset includes 7,715 QA pairs, augmented with 27,455 via zero-shot prompting.

Abstract

This article presents the first comprehensive study on designing a baseline extractive question-answering (QA) system for the Hindi tourism domain, with a specialized focus on the Varanasi-a cultural and spiritual hub renowned for its Bhakti-Bhaav (devotional ethos). Targeting ten tourism-centric subdomains-Ganga Aarti, Cruise, Food Court, Public Toilet, Kund, Museum, General, Ashram, Temple and Travel, the work addresses the absence of language-specific QA resources in Hindi for culturally nuanced applications. In this paper, a dataset comprising 7,715 Hindi QA pairs pertaining to Varanasi tourism was constructed and subsequently augmented with 27,455 pairs generated via Llama zero-shot prompting. We propose a framework leveraging foundation models-BERT and RoBERTa, fine-tuned using Supervised Fine-Tuning (SFT) and Low-Rank Adaptation (LoRA), to optimize parameter efficiency and task performance. Multiple variants of BERT, including pre-trained languages (e.g., Hindi-BERT), are evaluated to assess their suitability for low-resource domain-specific QA. Evaluation metrics - F1, BLEU, and ROUGE-L - highlight trade-offs between answer precision and linguistic fluency. Experiments demonstrate that LoRA-based fine-tuning achieves competitive performance (85.3\% F1) while reducing trainable parameters by 98\% compared to SFT, striking a balance between efficiency and accuracy. Comparative analysis across models reveals that RoBERTa with SFT outperforms BERT variants in capturing contextual nuances, particularly for culturally embedded terms (e.g., Aarti, Kund). This work establishes a foundational baseline for Hindi tourism QA systems, emphasizing the role of LORA in low-resource settings and underscoring the need for culturally contextualized NLP frameworks in the tourism domain.