Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spaces

TL;DR

This paper introduces a reinforcement learning method for multi-object search in continuous action spaces, enabling effective long-horizon exploration and zero-shot transfer to real-world environments with limited training data.

Contribution

It presents a novel RL approach that combines short-term and long-term reasoning without hierarchical complexity, excelling in continuous action spaces and generalizing to unseen environments.

Findings

Achieves high performance in continuous action spaces

Generalizes to unseen apartment environments with limited data

Successfully transfers policies to real-world office environments

Abstract

Recent advances in vision-based navigation and exploration have shown impressive capabilities in photorealistic indoor environments. However, these methods still struggle with long-horizon tasks and require large amounts of data to generalize to unseen environments. In this work, we present a novel reinforcement learning approach for multi-object search that combines short-term and long-term reasoning in a single model while avoiding the complexities arising from hierarchical structures. In contrast to existing multi-object search methods that act in granular discrete action spaces, our approach achieves exceptional performance in continuous action spaces. We perform extensive experiments and show that it generalizes to unseen apartment environments with limited data. Furthermore, we demonstrate zero-shot transfer of the learned policies to an office environment in real world experiments.