Semantic Instance Labeling Leveraging Hierarchical Segmentation

TL;DR

This paper introduces a hierarchical segmentation approach for indoor RGBD semantic labeling, improving classification accuracy by training on meaningful object-like segments rather than pixels or superpixels.

Contribution

It proposes a novel hierarchical segmentation method that enhances semantic labeling accuracy by focusing on object-conforming segments for classifier training.

Findings

Achieves state-of-the-art results on NYU V2 dataset

Hierarchical segmentation outperforms superpixel-based training

Improves semantic labeling for both general and object-specific classes

Abstract

Most of the approaches for indoor RGBD semantic la- beling focus on using pixels or superpixels to train a classi- fier. In this paper, we implement a higher level segmentation using a hierarchy of superpixels to obtain a better segmen- tation for training our classifier. By focusing on meaningful segments that conform more directly to objects, regardless of size, we train a random forest of decision trees as a clas- sifier using simple features such as the 3D size, LAB color histogram, width, height, and shape as specified by a his- togram of surface normals. We test our method on the NYU V2 depth dataset, a challenging dataset of cluttered indoor environments. Our experiments using the NYU V2 depth dataset show that our method achieves state of the art re- sults on both a general semantic labeling introduced by the dataset (floor, structure, furniture, and objects) and a more object specific semantic labeling. We show that training a classifier on a segmentation from a hierarchy of super pixels yields better results than training directly on super pixels, patches, or pixels as in previous work.