Temporal and Physical Reasoning for Perception-Based Robotic Manipulation

TL;DR

This paper improves robotic scene understanding by integrating physics simulation with pose estimation over time, enhancing accuracy and stability in cluttered, noisy environments for complex manipulation tasks.

Contribution

It introduces a method that enforces physical constraints over time to improve pose estimation accuracy and stability in robotic scene understanding.

Findings

Enhanced pose accuracy in simulated and real data

Successful application to block stacking task

Improved scene stability and physical consistency

Abstract

Accurate knowledge of object poses is crucial to successful robotic manipulation tasks, and yet most current approaches only work in laboratory settings. Noisy sensors and cluttered scenes interfere with accurate pose recognition, which is problematic especially when performing complex tasks involving object interactions. This is because most pose estimation algorithms focus only on estimating objects from a single frame, which means they lack continuity between frames. Further, they often do not consider resulting physical properties of the predicted scene such as intersecting objects or objects in unstable positions. In this work, we enhance the accuracy and stability of estimated poses for a whole scene by enforcing these physical constraints over time through the integration of a physics simulation. This allows us to accurately determine relationships between objects for a construction task. Scene parsing performance was evaluated on both simulated and real- world data. We apply our method to a real-world block stacking task, where the robot must build a tall tower of colored blocks.