Robotic manipulation of multiple objects as a POMDP

TL;DR

This paper models multi-object robotic manipulation under uncertainty as a POMDP, introducing an online particle filtering approach for adaptive planning in cluttered environments, validated through simulations and physical experiments.

Contribution

It presents a novel online POMDP method with particle filtering for multi-object manipulation, automatically adapting success probabilities and handling occlusions in real-time.

Findings

Multi-step POMDP planning outperforms greedy heuristics.

Online planning adapts system dynamics based on real experience.

Approach validated in simulation and physical experiments.

Abstract

This paper investigates manipulation of multiple unknown objects in a crowded environment. Because of incomplete knowledge due to unknown objects and occlusions in visual observations, object observations are imperfect and action success is uncertain, making planning challenging. We model the problem as a partially observable Markov decision process (POMDP), which allows a general reward based optimization objective and takes uncertainty in temporal evolution and partial observations into account. In addition to occlusion dependent observation and action success probabilities, our POMDP model also automatically adapts object specific action success probabilities. To cope with the changing system dynamics and performance constraints, we present a new online POMDP method based on particle filtering that produces compact policies. The approach is validated both in simulation and in physical experiments in a scenario of moving dirty dishes into a dishwasher. The results indicate that: 1) a greedy heuristic manipulation approach is not sufficient, multi-object manipulation requires multi-step POMDP planning, and 2) on-line planning is beneficial since it allows the adaptation of the system dynamics model based on actual experience.