TRansPose: Large-Scale Multispectral Dataset for Transparent Object

TL;DR

TRansPose is a comprehensive large-scale multispectral dataset combining RGB-D, thermal infrared images, and object poses to facilitate research on transparent object recognition under diverse real-world conditions.

Contribution

It introduces the first extensive multispectral dataset for transparent objects, including diverse object types, extensive annotations, and challenging scenarios to advance transparent object research.

Findings

Provides over 333,000 images and 4 million annotations.

Includes diverse scenarios like water-filled objects and cluttered environments.

Enables improved recognition of transparent objects using multispectral data.

Abstract

Transparent objects are encountered frequently in our daily lives, yet recognizing them poses challenges for conventional vision sensors due to their unique material properties, not being well perceived from RGB or depth cameras. Overcoming this limitation, thermal infrared cameras have emerged as a solution, offering improved visibility and shape information for transparent objects. In this paper, we present TRansPose, the first large-scale multispectral dataset that combines stereo RGB-D, thermal infrared (TIR) images, and object poses to promote transparent object research. The dataset includes 99 transparent objects, encompassing 43 household items, 27 recyclable trashes, 29 chemical laboratory equivalents, and 12 non-transparent objects. It comprises a vast collection of 333,819 images and 4,000,056 annotations, providing instance-level segmentation masks, ground-truth poses, and completed depth information. The data was acquired using a FLIR A65 thermal infrared (TIR) camera, two Intel RealSense L515 RGB-D cameras, and a Franka Emika Panda robot manipulator. Spanning 87 sequences, TRansPose covers various challenging real-life scenarios, including objects filled with water, diverse lighting conditions, heavy clutter, non-transparent or translucent containers, objects in plastic bags, and multi-stacked objects. TRansPose dataset can be accessed from the following link: https://sites.google.com/view/transpose-dataset