# Multi-view object pose distribution tracking for pre-grasp planning on mobile robots

## TL;DR

This paper introduces a method for tracking object poses from multiple camera views to help mobile robots plan grasps more efficiently and accurately.

## Contribution

A particle filter-based multi-view 6D pose tracking framework that fuses data from multiple cameras for improved accuracy and uncertainty estimation.

## Key findings

- The proposed framework outperforms previous methods in quantifying object pose and uncertainty.
- The method is effective for pre-grasp planning on mobile robots in real-world scenarios.
- A real-world benchmark dataset was created to evaluate the framework.

## Abstract

The ability to track the 6D pose distribution of an object while a mobile manipulator is still approaching it can enable the robot to pre-plan grasps, thereby improving both the time efficiency and robustness of mobile manipulation. However, tracking a 6D object pose distribution on approach can be challenging due to the limited view of the robot camera. In this study, we present a particle filter-based multi-view 6D pose distribution tracking framework that compensates for the limited view of the moving robot camera while it approaches the object by fusing observations from external stationary cameras in the environment. We extend the single-view pose distribution tracking framework (PoseRBPF) to fuse observations from external cameras. We model the object pose posterior as a multi-modal distribution and introduce techniques for fusion, re-sampling, and pose estimation from the tracked distribution to effectively handle noisy and conflicting observations from different cameras. To evaluate our framework, we also contribute a real-world benchmark dataset. Our experiments demonstrate that the proposed framework yields a more accurate quantification of object pose and associated uncertainty than previous research. Finally, we apply our framework for pre-grasp planning on mobile robots, demonstrating its practical utility.