Efficient Navigation Among Movable Obstacles using a Mobile Manipulator via Hierarchical Policy Learning

TL;DR

This paper introduces a hierarchical reinforcement learning framework enabling mobile manipulators to efficiently navigate among movable obstacles by combining obstacle property estimation with structured pushing strategies, resulting in improved success rates and reduced traversal times.

Contribution

The paper presents a novel HRL approach that integrates obstacle property estimation with structured pushing strategies for NAMO tasks, enhancing navigation efficiency.

Findings

Higher success rates in NAMO tasks

Shorter path lengths and goal-reaching times

Effective real-time obstacle property estimation

Abstract

We propose a hierarchical reinforcement learning (HRL) framework for efficient Navigation Among Movable Obstacles (NAMO) using a mobile manipulator. Our approach combines interaction-based obstacle property estimation with structured pushing strategies, facilitating the dynamic manipulation of unforeseen obstacles while adhering to a pre-planned global path. The high-level policy generates pushing commands that consider environmental constraints and path-tracking objectives, while the low-level policy precisely and stably executes these commands through coordinated whole-body movements. Comprehensive simulation-based experiments demonstrate improvements in performing NAMO tasks, including higher success rates, shortened traversed path length, and reduced goal-reaching times, compared to baselines. Additionally, ablation studies assess the efficacy of each component, while a qualitative analysis further validates the accuracy and reliability of the real-time obstacle property estimation.