Semantic Zone-Based Map Management for Stable AI-Integrated Mobile Robots

TL;DR

This paper introduces a semantic zone-based map management system that enhances the stability and efficiency of dense map utilization in memory-constrained AI-integrated mobile robots, demonstrated through large-scale simulations and real hardware tests.

Contribution

It proposes a novel semantic zone-based approach to prioritize spatially relevant map content, reducing memory usage and improving localization stability under memory constraints.

Findings

Improves throughput by 3.3 tokens/sec with Qwen3.5:0.8b.

Reduces latency by 21.7% compared to geometric strategies.

Eliminates out-of-memory failures and stalled execution under memory pressure.

Abstract

Recent advances in large AI models (VLMs and LLMs) and joint use of the 3D dense maps, enable mobile robots to provide more powerful and interactive services grounded in rich spatial context. However, deploying both heavy AI models and dense maps on edge robots is challenging under strict memory budgets. When the memory budget is exceeded, required keyframes may not be loaded in time, which can degrade the stability of position estimation and interfering model performance. We proposes a semantic zone-based map management approach to stabilize dense-map utilization under memory constraints. We associate keyframes with semantic indoor regions (e.g., rooms and corridors) and keyframe management at the semantic zone level prioritizes spatially relevant map content while respecting memory constraints. This reduces keyframe loading and unloading frequency and memory usage. We evaluate the proposed approach in large-scale simulated indoor environments and on an NVIDIA Jetson Orin Nano under concurrent SLAM-VLM execution. With Qwen3.5:0.8b, the proposed method improves throughput by 3.3 tokens/s and reduces latency by 21.7% relative to a geometric map-management strategy. Furthermore, while the geometric strategy suffers from out-of-memory failures and stalled execution under memory pressure, the proposed method eliminates both issues, preserving localization stability and enabling robust VLM operation. These results demonstrate that the proposed approach enables efficient dense map utilization for memory constrained, AI-integrated mobile robots. Code is available at: https://github.com/huichangs/rtabmap/tree/segment