QA-Dragon: Query-Aware Dynamic RAG System for Knowledge-Intensive Visual Question Answering

TL;DR

QA-Dragon is a novel query-aware dynamic retrieval system for knowledge-intensive visual question answering that effectively combines text and image retrieval for complex, multi-hop reasoning tasks, significantly improving accuracy.

Contribution

It introduces a domain router and a search router to dynamically select retrieval strategies, enabling multimodal, multi-turn, and multi-hop reasoning in VQA.

Findings

Outperforms baselines by 5.06% on single-source tasks

Achieves 6.35% improvement on multi-source tasks

Enhances reasoning performance in complex scenarios

Abstract

Retrieval-Augmented Generation (RAG) has been introduced to mitigate hallucinations in Multimodal Large Language Models (MLLMs) by incorporating external knowledge into the generation process, and it has become a widely adopted approach for knowledge-intensive Visual Question Answering (VQA). However, existing RAG methods typically retrieve from either text or images in isolation, limiting their ability to address complex queries that require multi-hop reasoning or up-to-date factual knowledge. To address this limitation, we propose QA-Dragon, a Query-Aware Dynamic RAG System for Knowledge-Intensive VQA. Specifically, QA-Dragon introduces a domain router to identify the query's subject domain for domain-specific reasoning, along with a search router that dynamically selects optimal retrieval strategies. By orchestrating both text and image search agents in a hybrid setup, our system supports multimodal, multi-turn, and multi-hop reasoning, enabling it to tackle complex VQA tasks effectively. We evaluate our QA-Dragon on the Meta CRAG-MM Challenge at KDD Cup 2025, where it significantly enhances the reasoning performance of base models under challenging scenarios. Our framework achieves substantial improvements in both answer accuracy and knowledge overlap scores, outperforming baselines by 5.06% on the single-source task, 6.35% on the multi-source task, and 5.03% on the multi-turn task.