Efficient Q-Learning over Visit Frequency Maps for Multi-agent Exploration of Unknown Environments

TL;DR

This paper introduces an efficient, compact multi-agent exploration method using an integrated visit frequency map, improving communication efficiency and adaptability in unknown environments.

Contribution

The paper proposes a novel integrated visit frequency map and a multi-agent information exchange scheme, enhancing efficiency and scalability over traditional visit frequency maps.

Findings

Achieves similar performance to VFM with lower bandwidth.

Generalizes well to various multi-agent and real-world environments.

Reduces map size while maintaining exploration effectiveness.

Abstract

The robot exploration task has been widely studied with applications spanning from novel environment mapping to item delivery. For some time-critical tasks, such as rescue catastrophes, the agent is required to explore as efficiently as possible. Recently, Visit Frequency-based map representation achieved great success in such scenarios by discouraging repetitive visits with a frequency-based penalty. However, its relatively large size and single-agent settings hinder its further development. In this context, we propose Integrated Visit Frequency Map, which encodes identical information as Visit Frequency Map with a more compact size, and a visit frequency-based multi-agent information exchange and control scheme that is able to accommodate both representations. Through tests in diverse settings, the results indicate our proposed methods can achieve a comparable level of performance of VFM with lower bandwidth requirements and generalize well to different multi-agent setups including real-world environments.