Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning

TL;DR

This paper presents a model-free constrained reinforcement learning approach enabling a mobile manipulator to effectively push and reorient unknown objects with high success rates, demonstrating robustness and adaptability in real-world scenarios.

Contribution

It introduces a novel RL-based controller for mobile manipulators that handles uncertainties and learns contact-rich pushing behaviors without prior object models.

Findings

Success rate of 91.35% in simulation

At least 80% success rate on hardware

Robustness against diverse object properties

Abstract

Non-prehensile pushing to move and reorient objects to a goal is a versatile loco-manipulation skill. In the real world, the object's physical properties and friction with the floor contain significant uncertainties, which makes the task challenging for a mobile manipulator. In this paper, we develop a learning-based controller for a mobile manipulator to move an unknown object to a desired position and yaw orientation through a sequence of pushing actions. The proposed controller for the robotic arm and the mobile base motion is trained using a constrained Reinforcement Learning (RL) formulation. We demonstrate its capability in experiments with a quadrupedal robot equipped with an arm. The learned policy achieves a success rate of 91.35% in simulation and at least 80% on hardware in challenging scenarios. Through our extensive hardware experiments, we show that the approach demonstrates high robustness against unknown objects of different masses, materials, sizes, and shapes. It reactively discovers the pushing location and direction, thus achieving contact-rich behavior while observing only the pose of the object. Additionally, we demonstrate the adaptive behavior of the learned policy towards preventing the object from toppling.