Achieving RGB-D level Segmentation Performance from a Single ToF Camera

TL;DR

This paper demonstrates that a single ToF camera capturing IR and depth images can achieve segmentation accuracy comparable to traditional RGB-D cameras by using depth-specific convolutions within a multi-task learning framework.

Contribution

The authors introduce a novel method for fusing IR and depth data from a single ToF camera, achieving RGB-D level segmentation performance.

Findings

Achieved competitive segmentation accuracy with a single ToF camera.

Demonstrated effectiveness of depth-specific convolutions in multi-task learning.

Validated approach on an in-car segmentation dataset.

Abstract

Depth is a very important modality in computer vision, typically used as complementary information to RGB, provided by RGB-D cameras. In this work, we show that it is possible to obtain the same level of accuracy as RGB-D cameras on a semantic segmentation task using infrared (IR) and depth images from a single Time-of-Flight (ToF) camera. In order to fuse the IR and depth modalities of the ToF camera, we introduce a method utilizing depth-specific convolutions in a multi-task learning framework. In our evaluation on an in-car segmentation dataset, we demonstrate the competitiveness of our method against the more costly RGB-D approaches.