MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering

TL;DR

MARA is a novel framework for multimodal document question answering that uses query-adaptive retrieval and generation mechanisms to improve relevance and answer quality.

Contribution

It introduces query-adaptive retrieval and generation components, addressing limitations of static evidence selection in multimodal QA.

Findings

MARA outperforms existing SOTA methods on six benchmarks.

It improves retrieval relevance and answer accuracy.

The adaptive mechanisms enhance handling complex multimodal documents.

Abstract

Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures. While retrieval-augmented generation (RAG) has shown strong performance in text-based QA, its extensions to multimodal documents remain underexplored and face significant limitations. Specifically, current approaches rely on query-agnostic document representations that overlook salient content and use static top-k evidence selection, which fails to adapt to the uncertain distribution of relevant information. To address these limitations, we propose the Multimodal Adaptive Retrieval-Augmented (MARA) framework, which introduces query-adaptive mechanisms to both retrieval and generation. MARA consists of two components: a Query-Aligned Region Encoder that builds multi-level document representations and reweights them based on query relevance to improve retrieval precision; and a Self-Reflective Evidence Controller that monitors evidence sufficiency during generation and adaptively incorporates content from lower-ranked sources using a sliding-window strategy. Experiments on six multimodal QA benchmarks demonstrate that MARA consistently improves retrieval relevance and answer quality over existing SOTA method.