Plenoptic Monte Carlo Object Localization for Robot Grasping under Layered Translucency

TL;DR

This paper introduces Plenoptic Monte Carlo Localization (PMCL), a novel method for robot perception that localizes objects with translucent materials using light-field data, enabling improved grasping in complex environments.

Contribution

The paper proposes a new depth descriptor, the Depth Likelihood Volume, and integrates it into a Monte Carlo localization framework for handling translucency in robotic perception.

Findings

Successfully localizes translucent objects for robotic grasping

Uses light-field camera data to improve perception accuracy

Demonstrates manipulation of objects behind translucent layers

Abstract

In order to fully function in human environments, robot perception will need to account for the uncertainty caused by translucent materials. Translucency poses several open challenges in the form of transparent objects (e.g., drinking glasses), refractive media (e.g., water), and diffuse partial occlusions (e.g., objects behind stained glass panels). This paper presents Plenoptic Monte Carlo Localization (PMCL) as a method for localizing object poses in the presence of translucency using plenoptic (light-field) observations. We propose a new depth descriptor, the Depth Likelihood Volume (DLV), and its use within a Monte Carlo object localization algorithm. We present results of localizing and manipulating objects with translucent materials and objects occluded by layers of translucency. Our PMCL implementation uses observations from a Lytro first generation light field camera to allow a Michigan Progress Fetch robot to perform grasping.