Knowledge Distillation for Semantic Segmentation: A Label Space Unification Approach

TL;DR

This paper introduces a knowledge distillation method that unifies label spaces across multiple datasets for semantic segmentation, improving model performance in urban and off-road driving scenarios.

Contribution

It proposes a novel label space unification approach using pseudo-labels generated by a teacher model to enhance training across diverse datasets.

Findings

Student models outperform teachers on multiple datasets.

Created the largest combined datasets for autonomous driving.

Effective in urban and off-road domains.

Abstract

An increasing number of datasets sharing similar domains for semantic segmentation have been published over the past few years. But despite the growing amount of overall data, it is still difficult to train bigger and better models due to inconsistency in taxonomy and/or labeling policies of different datasets. To this end, we propose a knowledge distillation approach that also serves as a label space unification method for semantic segmentation. In short, a teacher model is trained on a source dataset with a given taxonomy, then used to pseudo-label additional data for which ground truth labels of a related label space exist. By mapping the related taxonomies to the source taxonomy, we create constraints within which the model can predict pseudo-labels. Using the improved pseudo-labels we train student models that consistently outperform their teachers in two challenging domains, namely urban and off-road driving. Our ground truth-corrected pseudo-labels span over 12 and 7 public datasets with 388.230 and 18.558 images for the urban and off-road domains, respectively, creating the largest compound datasets for autonomous driving to date.