BanglaMedQA and BanglaMMedBench: Evaluating Retrieval-Augmented Generation Strategies for Bangla Biomedical Question Answering

TL;DR

This paper introduces BanglaMedQA and BanglaMMedBench datasets to evaluate retrieval-augmented generation strategies for Bangla biomedical question answering, demonstrating that Agentic RAG achieves high accuracy and improved reasoning in low-resource language medical AI.

Contribution

It presents the first large-scale Bangla biomedical MCQ datasets and benchmarks various RAG strategies, including a novel Agentic RAG pipeline integrating textbook and web retrieval.

Findings

Agentic RAG achieved 89.54% accuracy with GPT-OSS-120B.

The approach outperformed other RAG configurations in factual accuracy.

Integration of textbook OCR and dynamic retrieval strategies improved reasoning quality.

Abstract

Developing accurate biomedical Question Answering (QA) systems in low-resource languages remains a major challenge, limiting equitable access to reliable medical knowledge. This paper introduces BanglaMedQA and BanglaMMedBench, the first large-scale Bangla biomedical Multiple Choice Question (MCQ) datasets designed to evaluate reasoning and retrieval in medical artificial intelligence (AI). The study applies and benchmarks several Retrieval-Augmented Generation (RAG) strategies, including Traditional, Zero-Shot Fallback, Agentic, Iterative Feedback, and Aggregate RAG, combining textbook-based and web retrieval with generative reasoning to improve factual accuracy. A key novelty lies in integrating a Bangla medical textbook corpus through Optical Character Recognition (OCR) and implementing an Agentic RAG pipeline that dynamically selects between retrieval and reasoning strategies. Experimental results show that the Agentic RAG achieved the highest accuracy 89.54% with openai/gpt-oss-120b, outperforming other configurations and demonstrating superior rationale quality. These findings highlight the potential of RAG-based methods to enhance the reliability and accessibility of Bangla medical QA, establishing a foundation for future research in multilingual medical artificial intelligence.