SAM 3D for 3D Object Reconstruction from Remote Sensing Images

TL;DR

This paper evaluates SAM 3D, a general-purpose foundation model, for monocular 3D building reconstruction from remote sensing images, showing its advantages over existing methods and exploring its extension to urban scene modeling.

Contribution

First systematic evaluation of SAM 3D for remote sensing 3D reconstruction, benchmarking against TRELLIS, and extending its application to urban scene modeling.

Findings

SAM 3D yields more coherent roof geometry.

SAM 3D produces sharper boundaries.

Extension to urban scenes is feasible.

Abstract

Monocular 3D building reconstruction from remote sensing imagery is essential for scalable urban modeling, yet existing methods often require task-specific architectures and intensive supervision. This paper presents the first systematic evaluation of SAM 3D, a general-purpose image-to-3D foundation model, for monocular remote sensing building reconstruction. We benchmark SAM 3D against TRELLIS on samples from the NYC Urban Dataset, employing Frechet Inception Distance (FID) and CLIP-based Maximum Mean Discrepancy (CMMD) as evaluation metrics. Experimental results demonstrate that SAM 3D produces more coherent roof geometry and sharper boundaries compared to TRELLIS. We further extend SAM 3D to urban scene reconstruction through a segment-reconstruct-compose pipeline, demonstrating its potential for urban scene modeling. We also analyze practical limitations and discuss future research directions. These findings provide practical guidance for deploying foundation models in urban 3D reconstruction and motivate future integration of scene-level structural priors.