Glass Surface Segmentation with an RGB-D Camera via Weighted Feature Fusion for Service Robots

TL;DR

This paper introduces a novel Weighted Feature Fusion module for improved glass surface segmentation using RGB-D cameras, along with a new dataset, enhancing accuracy and robustness for service robot applications.

Contribution

The paper presents a dynamic feature fusion module and a comprehensive RGB-D dataset, advancing glass segmentation techniques for service robots.

Findings

WFF improves segmentation accuracy and robustness.

Achieves 7.49% improvement in boundary IoU.

Provides a new benchmark dataset for glass segmentation.

Abstract

We address the problem of glass surface segmentation with an RGB-D camera, with a focus on effectively fusing RGB and depth information. To this end, we propose a Weighted Feature Fusion (WFF) module that dynamically and adaptively combines RGB and depth features to tackle issues such as transparency, reflections, and occlusions. This module can be seamlessly integrated with various deep neural network backbones as a plug-and-play solution. Additionally, we introduce the MJU-Glass dataset, a comprehensive RGB-D dataset collected by a service robot navigating real-world environments, providing a valuable benchmark for evaluating segmentation models. Experimental results show significant improvements in segmentation accuracy and robustness, with the WFF module enhancing performance in both mean Intersection over Union (mIoU) and boundary IoU (bIoU), achieving a 7.49% improvement in bIoU when integrated with PSPNet. The proposed module and dataset provide a robust framework for advancing glass surface segmentation in robotics and reducing the risk of collisions with glass objects.