LIT: Light-field Inference of Transparency for Refractive Object Localization

TL;DR

This paper introduces LIT, a novel two-stage light-field based method for accurately estimating the pose of transparent objects, addressing challenges posed by transparency and complex lighting conditions.

Contribution

LIT combines light-field sensing with photorealistic rendering and deep learning to improve transparent object pose estimation, and introduces the ProLIT dataset for benchmarking.

Findings

LIT outperforms existing pose estimation methods on the ProLIT dataset.

The method effectively captures transparent material properties using light-field imagery.

LIT achieves robust depth and pose estimation for transparent objects.

Abstract

Translucency is prevalent in everyday scenes. As such, perception of transparent objects is essential for robots to perform manipulation. Compared with texture-rich or texture-less Lambertian objects, transparency induces significant uncertainty on object appearances. Ambiguity can be due to changes in lighting, viewpoint, and backgrounds, each of which brings challenges to existing object pose estimation algorithms. In this work, we propose LIT, a two-stage method for transparent object pose estimation using light-field sensing and photorealistic rendering. LIT employs multiple filters specific to light-field imagery in deep networks to capture transparent material properties, with robust depth and pose estimators based on generative sampling. Along with the LIT algorithm, we introduce the light-field transparent object dataset ProLIT for the tasks of recognition, localization and pose estimation. With respect to this ProLIT dataset, we demonstrate that LIT can outperform both state-of-the-art end-to-end pose estimation methods and a generative pose estimator on transparent objects.