Learning to Explore by Reinforcement over High-Level Options

TL;DR

This paper introduces a reinforcement learning approach with high-level options for efficient 3D environment exploration, combining look-around and frontier navigation behaviors to improve coverage and efficiency.

Contribution

It proposes a novel option-critic framework integrating macro-actions and path-planning for improved exploration in 3D environments.

Findings

Achieves higher coverage than competing methods

Demonstrates better efficiency in exploration tasks

Validated on two public 3D datasets

Abstract

Autonomous 3D environment exploration is a fundamental task for various applications such as navigation. The goal of exploration is to investigate a new environment and build its occupancy map efficiently. In this paper, we propose a new method which grants an agent two intertwined options of behaviors: "look-around" and "frontier navigation". This is implemented by an option-critic architecture and trained by reinforcement learning algorithms. In each timestep, an agent produces an option and a corresponding action according to the policy. We also take advantage of macro-actions by incorporating classic path-planning techniques to increase training efficiency. We demonstrate the effectiveness of the proposed method on two publicly available 3D environment datasets and the results show our method achieves higher coverage than competing techniques with better efficiency.