Automated Label Unification for Multi-Dataset Semantic Segmentation with GNNs

TL;DR

This paper introduces a graph neural network-based method to automatically unify label spaces across multiple datasets, enabling more effective multi-dataset semantic segmentation without manual reannotation, leading to improved performance.

Contribution

A novel GNN-based approach for automatic label space unification that simplifies multi-dataset training and enhances segmentation accuracy.

Findings

Outperforms existing multi-dataset training methods on seven datasets

Achieves state-of-the-art results on WildDash 2 benchmark

Enables seamless multi-dataset training without manual label reconciliation

Abstract

Deep supervised models possess significant capability to assimilate extensive training data, thereby presenting an opportunity to enhance model performance through training on multiple datasets. However, conflicts arising from different label spaces among datasets may adversely affect model performance. In this paper, we propose a novel approach to automatically construct a unified label space across multiple datasets using graph neural networks. This enables semantic segmentation models to be trained simultaneously on multiple datasets, resulting in performance improvements. Unlike existing methods, our approach facilitates seamless training without the need for additional manual reannotation or taxonomy reconciliation. This significantly enhances the efficiency and effectiveness of multi-dataset segmentation model training. The results demonstrate that our method significantly outperforms other multi-dataset training methods when trained on seven datasets simultaneously, and achieves state-of-the-art performance on the WildDash 2 benchmark.