SVII-3D: Advancing Roadside Infrastructure Inventory with Decimeter-level 3D Localization and Comprehension from Sparse Street Imagery

TL;DR

SVII-3D is a comprehensive framework that achieves decimeter-level 3D localization and detailed asset understanding from sparse street imagery, enabling scalable and accurate digital twin creation for smart cities.

Contribution

It introduces a novel unified approach combining open-set detection, spatial-attention matching, geometry refinement, and vision-language modeling for infrastructure digitization from sparse data.

Findings

Significantly improves asset identification accuracy.

Reduces localization errors to decimeter-level.

Enhances fine-grained operational state diagnosis.

Abstract

The automated creation of digital twins and precise asset inventories is a critical task in smart city construction and facility lifecycle management. However, utilizing cost-effective sparse imagery remains challenging due to limited robustness, inaccurate localization, and a lack of fine-grained state understanding. To address these limitations, SVII-3D, a unified framework for holistic asset digitization, is proposed. First, LoRA fine-tuned open-set detection is fused with a spatial-attention matching network to robustly associate observations across sparse views. Second, a geometry-guided refinement mechanism is introduced to resolve structural errors, achieving precise decimeter-level 3D localization. Third, transcending static geometric mapping, a Vision-Language Model agent leveraging multi-modal prompting is incorporated to automatically diagnose fine-grained operational states. Experiments demonstrate that SVII-3D significantly improves identification accuracy and minimizes localization errors. Consequently, this framework offers a scalable, cost-effective solution for high-fidelity infrastructure digitization, effectively bridging the gap between sparse perception and automated intelligent maintenance.