On Recognizing Transparent Objects in Domestic Environments Using Fusion of Multiple Sensor Modalities

TL;DR

This paper introduces a sensor fusion approach that combines multiple modalities to improve recognition of transparent and reflective objects in domestic environments, addressing limitations of existing methods.

Contribution

It presents a novel fusion technique that integrates specular reflectance and other cues to enhance recognition of challenging household objects.

Findings

Significant increase in recognition robustness for transparent objects

Extended recognition capabilities to diffuse, reflective, and transparent materials

Improved performance over existing state-of-the-art methods

Abstract

Current object recognition methods fail on object sets that include both diffuse, reflective and transparent materials, although they are very common in domestic scenarios. We show that a combination of cues from multiple sensor modalities, including specular reflectance and unavailable depth information, allows us to capture a larger subset of household objects by extending a state of the art object recognition method. This leads to a significant increase in robustness of recognition over a larger set of commonly used objects.