DM$^3$-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

TL;DR

DM$^3$-Nav is a decentralized multi-agent system enabling semantic navigation with multimodal, open-vocabulary goals, operating without central coordination and validated in both simulation and real-world environments.

Contribution

The paper introduces a fully decentralized multi-agent navigation system supporting multimodal goals, with implicit task allocation and real-world validation.

Findings

Achieves comparable or better performance than centralized baselines.

Operates successfully in real-world office environment using onboard sensing.

Eliminates single points of failure inherent in centralized systems.

Abstract

We present DM$^3$-Nav, a fully decentralized multi-agent semantic navigation system supporting multimodal open-vocabulary goal specification and multi-object missions. In our setting, decentralization implies operation without a central coordinator, global map aggregation, or shared global state at runtime. Robots operate autonomously and coordinate through ad-hoc pairwise communication, exchanging local maps, goal status, and navigation intent without synchronization. An implicit task allocation mechanism combining intent broadcasting and distance-weighted frontier selection reduces redundant exploration while preserving decentralized operation. Evaluations on HM3DSem scenes using the HM3Dv0.2 and GOAT-Bench datasets demonstrate that DM$^3$-Nav matches or exceeds centralized and shared-map baselines while eliminating single points of failure inherent in centralized architectures. Finally, we validate our approach in a real-world office environment using two mobile robots, demonstrating successful deployment relying entirely on onboard sensing and computation. A video of our real-world experiments is available online: https://drive.google.com/file/d/1QiUSCn5rIvtuTUqtuXLPgmt6S8x9-MCZ/view?usp=drive_link