Automatic universal taxonomies for multi-domain semantic segmentation

TL;DR

This paper introduces an automatic method to create universal taxonomies for multi-domain semantic segmentation, enabling models to handle incompatible labels across datasets and improve generalization.

Contribution

It proposes an iterative dataset integration approach that detects label relationships and supports learning sub-class logits, advancing multi-domain segmentation.

Findings

Achieves competitive generalization performance on standard datasets

Effectively detects subset-superset relationships between labels

Supports learning of sub-class logits through super-classes

Abstract

Training semantic segmentation models on multiple datasets has sparked a lot of recent interest in the computer vision community. This interest has been motivated by expensive annotations and a desire to achieve proficiency across multiple visual domains. However, established datasets have mutually incompatible labels which disrupt principled inference in the wild. We address this issue by automatic construction of universal taxonomies through iterative dataset integration. Our method detects subset-superset relationships between dataset-specific labels, and supports learning of sub-class logits by treating super-classes as partial labels. We present experiments on collections of standard datasets and demonstrate competitive generalization performance with respect to previous work.