MGA-VQA: Secure and Interpretable Graph-Augmented Visual Question Answering with Memory-Guided Protection Against Unauthorized Knowledge Use

TL;DR

MGA-VQA is a novel multi-modal framework for Document Visual Question Answering that enhances interpretability, reasoning, and efficiency by integrating graph reasoning, memory, and question-guided compression, outperforming prior models across multiple benchmarks.

Contribution

It introduces a graph-augmented, memory-guided approach for DocVQA that improves interpretability and reasoning capabilities over existing black-box models.

Findings

Achieves superior accuracy on six DocVQA benchmarks.

Enhances interpretability with graph-based decision pathways.

Demonstrates improved efficiency in high-resolution document processing.

Abstract

Document Visual Question Answering (DocVQA) requires models to jointly understand textual semantics, spatial layout, and visual features. Current methods struggle with explicit spatial relationship modeling, inefficiency with high-resolution documents, multi-hop reasoning, and limited interpretability. We propose MGA-VQA, a multi-modal framework that integrates token-level encoding, spatial graph reasoning, memory-augmented inference, and question-guided compression. Unlike prior black-box models, MGA-VQA introduces interpretable graph-based decision pathways and structured memory access for enhanced reasoning transparency. Evaluation across six benchmarks (FUNSD, CORD, SROIE, DocVQA, STE-VQA, and RICO) demonstrates superior accuracy and efficiency, with consistent improvements in both answer prediction and spatial localization.