Manipulate-to-Navigate: Reinforcement Learning with Visual Affordances and Manipulability Priors

TL;DR

This paper introduces a reinforcement learning approach that combines manipulability priors and affordance maps to enable robots to manipulate obstacles and navigate in dynamic environments, demonstrated on simulation and real robots.

Contribution

It presents a novel RL-based method integrating manipulability priors and affordance maps for manipulate-to-navigate tasks, with new simulation benchmarks and real-world transfer.

Findings

The approach enables effective obstacle manipulation and navigation in dynamic scenarios.

The method reduces exploration by focusing on feasible actions.

Successful transfer of policies from simulation to real robots.

Abstract

Mobile manipulation in dynamic environments is challenging due to movable obstacles blocking the robot's path. Traditional methods, which treat navigation and manipulation as separate tasks, often fail in such 'manipulate-to-navigate' scenarios, as obstacles must be removed before navigation. In these cases, active interaction with the environment is required to clear obstacles while ensuring sufficient space for movement. To address the manipulate-to-navigate problem, we propose a reinforcement learning-based approach for learning manipulation actions that facilitate subsequent navigation. Our method combines manipulability priors to focus the robot on high manipulability body positions with affordance maps for selecting high-quality manipulation actions. By focusing on feasible and meaningful actions, our approach reduces unnecessary exploration and allows the robot to learn manipulation strategies more effectively. We present two new manipulate-to-navigate simulation tasks called Reach and Door with the Boston Dynamics Spot robot. The first task tests whether the robot can select a good hand position in the target area such that the robot base can move effectively forward while keeping the end effector position fixed. The second task requires the robot to move a door aside in order to clear the navigation path. Both of these tasks need first manipulation and then navigating the base forward. Results show that our method allows a robot to effectively interact with and traverse dynamic environments. Finally, we transfer the learned policy to a real Boston Dynamics Spot robot, which successfully performs the Reach task.