Indoor Semantic Segmentation using depth information

TL;DR

This paper presents a multiscale convolutional network approach for indoor semantic segmentation using RGB-D data, achieving state-of-the-art accuracy and enabling real-time scene labeling on hardware like FPGA.

Contribution

It introduces a deep learning method that learns features directly from RGB-D inputs, surpassing previous hand-crafted feature approaches.

Findings

Achieved 64.5% accuracy on NYU-v2 dataset

Demonstrated real-time scene labeling potential

Outperformed existing methods in indoor segmentation

Abstract

This work addresses multi-class segmentation of indoor scenes with RGB-D inputs. While this area of research has gained much attention recently, most works still rely on hand-crafted features. In contrast, we apply a multiscale convolutional network to learn features directly from the images and the depth information. We obtain state-of-the-art on the NYU-v2 depth dataset with an accuracy of 64.5%. We illustrate the labeling of indoor scenes in videos sequences that could be processed in real-time using appropriate hardware such as an FPGA.