AREA3D: Active Reconstruction Agent with Unified Feed-Forward 3D Perception and Vision-Language Guidance

TL;DR

AREA3D introduces an active 3D reconstruction agent that combines feed-forward models and vision-language guidance to improve scene reconstruction accuracy efficiently, especially with limited views.

Contribution

The paper presents a novel active reconstruction framework that decouples uncertainty estimation from the reconstructor and integrates semantic guidance for better viewpoint selection.

Findings

Achieves state-of-the-art accuracy in sparse-view scenarios

Effectively estimates view uncertainty without online optimization

Utilizes vision-language guidance for diverse and informative viewpoints

Abstract

Active 3D reconstruction enables an agent to autonomously select viewpoints to efficiently obtain accurate and complete scene geometry, rather than passively reconstructing scenes from pre-collected images. However, existing active reconstruction methods often rely on hand-crafted geometric heuristics, which can lead to redundant observations without substantially improving reconstruction quality. To address this limitation, we propose AREA3D, an active reconstruction agent that leverages feed-forward 3D reconstruction models and vision-language guidance. Our framework decouples view-uncertainty modeling from the underlying feed-forward reconstructor, enabling precise uncertainty estimation without expensive online optimization. In addition, an integrated vision-language model provides high-level semantic guidance, encouraging informative and diverse viewpoints beyond purely geometric cues. Extensive experiments on both scene-level and object-level benchmarks demonstrate that AREA3D achieves state-of-the-art reconstruction accuracy, particularly in the sparse-view regime. Code will be made available at: https://github.com/TianlingXu/AREA3D .