ALDEN: Reinforcement Learning for Active Navigation and Evidence Gathering in Long Documents

TL;DR

ALDEN is a reinforcement learning framework that enables interactive, goal-directed navigation and reasoning in long, visually complex documents, surpassing passive reading methods in accuracy and efficiency.

Contribution

ALDEN introduces a novel active navigation approach with a fetch action, a rule-based reward system, and a visual-semantic anchoring mechanism for stable training of VLMs on long documents.

Findings

Achieves state-of-the-art results on five long-document benchmarks.

Effectively stabilizes training with visual-semantic anchoring.

Demonstrates improved navigation and reasoning in complex documents.

Abstract

Vision-language models (VLMs) excel at interpreting text-rich images but struggle with long, visually complex documents that demand analysis and integration of information spread across multiple pages. Existing approaches typically rely on fixed reasoning templates or rigid pipelines, which force VLMs into a passive role and hinder both efficiency and generalization. We present Active Long-DocumEnt Navigation (ALDEN), a multi-turn reinforcement learning framework that fine-tunes VLMs as interactive agents capable of actively navigating long, visually rich documents. ALDEN introduces a novel fetch action that directly accesses the page by index, complementing the classic search action and better exploiting document structure. For dense process supervision and efficient training, we propose a rule-based cross-level reward that provides both turn- and token-level signals. To address the empirically observed training instability caused by numerous visual tokens from long documents, we further propose a visual-semantic anchoring mechanism that applies a dual-path KL-divergence constraint to stabilize visual and textual representations separately during training. Trained on a corpus constructed from three open-source datasets, ALDEN achieves state-of-the-art performance on five long-document benchmarks. Overall, ALDEN marks a step beyond passive document reading toward agents that autonomously navigate and reason across long, visually rich documents, offering a robust path to more accurate and efficient long-document understanding.