Physics-Based Object 6D-Pose Estimation during Non-Prehensile Manipulation

TL;DR

This paper introduces a physics-based particle filtering method for accurately tracking the 6D pose of objects during non-prehensile manipulation, leveraging robot controls and visual data to improve robustness and precision.

Contribution

The paper presents a novel physics-informed particle filtering approach that integrates robot control data with visual observations for improved 6D object pose estimation during manipulation.

Findings

Physics-based predictions improve pose accuracy.

Method outperforms image-only and constant-velocity baselines.

Effective even with occlusions or poor visibility.

Abstract

We propose a method to track the 6D pose of an object over time, while the object is under non-prehensile manipulation by a robot. At any given time during the manipulation of the object, we assume access to the robot joint controls and an image from a camera. We use the robot joint controls to perform a physics-based prediction of how the object might be moving. We then combine this prediction with the observation coming from the camera, to estimate the object pose as accurately as possible. We use a particle filtering approach to combine the control information with the visual information. We compare the proposed method with two baselines: (i) using only an image-based pose estimation system at each time-step, and (ii) a particle filter which does not perform the computationally expensive physics predictions, but assumes the object moves with constant velocity. Our results show that making physics-based predictions is worth the computational cost, resulting in more accurate tracking, and estimating object pose even when the object is not clearly visible to the camera.