Multi-domain semantic segmentation with overlapping labels

TL;DR

This paper introduces a probabilistic loss-based method for multi-domain semantic segmentation that effectively handles overlapping labels across datasets, enabling improved generalization and learning of unlabeled visual concepts.

Contribution

It presents a novel principled approach for training on datasets with incompatible and overlapping labels using partial labels and probabilistic loss functions.

Findings

Achieves competitive or state-of-the-art performance on multiple datasets.

Demonstrates improved cross-dataset generalization.

Learns visual concepts not explicitly labeled in training data.

Abstract

Deep supervised models have an unprecedented capacity to absorb large quantities of training data. Hence, training on many datasets becomes a method of choice towards graceful degradation in unusual scenes. Unfortunately, different datasets often use incompatible labels. For instance, the Cityscapes road class subsumes all driving surfaces, while Vistas defines separate classes for road markings, manholes etc. We address this challenge by proposing a principled method for seamless learning on datasets with overlapping classes based on partial labels and probabilistic loss. Our method achieves competitive within-dataset and cross-dataset generalization, as well as ability to learn visual concepts which are not separately labeled in any of the training datasets. Experiments reveal competitive or state-of-the-art performance on two multi-domain dataset collections and on the WildDash 2 benchmark.