Have We Mastered Scale in Deep Monocular Visual SLAM? The ScaleMaster Dataset and Benchmark

TL;DR

This paper introduces the ScaleMaster Dataset and benchmark to evaluate and analyze the robustness of deep monocular visual SLAM systems in large-scale indoor environments, highlighting their scale inconsistency issues.

Contribution

The paper presents the first benchmark specifically designed to assess scale consistency in challenging large-scale indoor SLAM scenarios, along with a systematic analysis of current systems' vulnerabilities.

Findings

Deep SLAM systems perform well on existing benchmarks but struggle with scale in large environments.

Scale inconsistency causes significant failures in realistic indoor scenarios.

The new benchmark enables more comprehensive evaluation of scale robustness.

Abstract

Recent advances in deep monocular visual Simultaneous Localization and Mapping (SLAM) have achieved impressive accuracy and dense reconstruction capabilities, yet their robustness to scale inconsistency in large-scale indoor environments remains largely unexplored. Existing benchmarks are limited to room-scale or structurally simple settings, leaving critical issues of intra-session scale drift and inter-session scale ambiguity insufficiently addressed. To fill this gap, we introduce the ScaleMaster Dataset, the first benchmark explicitly designed to evaluate scale consistency under challenging scenarios such as multi-floor structures, long trajectories, repetitive views, and low-texture regions. We systematically analyze the vulnerability of state-of-the-art deep monocular visual SLAM systems to scale inconsistency, providing both quantitative and qualitative evaluations. Crucially, our analysis extends beyond traditional trajectory metrics to include a direct map-to-map quality assessment using metrics like Chamfer distance against high-fidelity 3D ground truth. Our results reveal that while recent deep monocular visual SLAM systems demonstrate strong performance on existing benchmarks, they suffer from severe scale-related failures in realistic, large-scale indoor environments. By releasing the ScaleMaster dataset and baseline results, we aim to establish a foundation for future research toward developing scale-consistent and reliable visual SLAM systems.