Learning to Ground Objects for Robot Task and Motion Planning

TL;DR

This paper introduces TMOC, a novel object-centric TAMP algorithm that enables robots to learn object properties and ground objects using a physics engine, improving planning in complex, real-world scenarios.

Contribution

The paper presents TMOC, a new grounded TAMP algorithm that learns object properties and grounding through simulation and real robot experiments, addressing modeling challenges.

Findings

TMOC outperforms existing methods in simulation and real robot tasks.

TMOC effectively learns object properties like size and weight.

Enhanced planning in complex multi-object interactions.

Abstract

Task and motion planning (TAMP) algorithms have been developed to help robots plan behaviors in discrete and continuous spaces. Robots face complex real-world scenarios, where it is hardly possible to model all objects or their physical properties for robot planning (e.g., in kitchens or shopping centers). In this paper, we define a new object-centric TAMP problem, where the TAMP robot does not know object properties (e.g., size and weight of blocks). We then introduce Task-Motion Object-Centric planning ({\bf TMOC}), a grounded TAMP algorithm that learns to ground objects and their physical properties with a physics engine. TMOC is particularly useful for those tasks that involve dynamic complex robot-multi-object interactions that can hardly be modeled beforehand. We have demonstrated and evaluated TMOC in simulation and using a real robot. Results show that TMOC outperforms competitive baselines from the literature in cumulative utility.