Evaluation of Multimodal Semantic Segmentation using RGB-D Data

TL;DR

This paper evaluates a state-of-the-art multimodal semantic segmentation approach using RGB-D data across diverse datasets, proposing new multi-dataset learning strategies to enhance detection of unseen objects in outdoor scenes.

Contribution

It provides a comprehensive evaluation of multimodal semantic segmentation methods and introduces novel multi-dataset learning strategies for improved unseen object recognition.

Findings

Effective fusion of RGB and depth data improves segmentation accuracy.

Multi-dataset learning enhances recognition of unseen objects.

Extensive experiments validate the robustness of the proposed approach.

Abstract

Our goal is to develop stable, accurate, and robust semantic scene understanding methods for wide-area scene perception and understanding, especially in challenging outdoor environments. To achieve this, we are exploring and evaluating a range of related technology and solutions, including AI-driven multimodal scene perception, fusion, processing, and understanding. This work reports our efforts on the evaluation of a state-of-the-art approach for semantic segmentation with multiple RGB and depth sensing data. We employ four large datasets composed of diverse urban and terrain scenes and design various experimental methods and metrics. In addition, we also develop new strategies of multi-datasets learning to improve the detection and recognition of unseen objects. Extensive experiments, implementations, and results are reported in the paper.