Synthetic Convolutional Features for Improved Semantic Segmentation

TL;DR

This paper introduces a novel method for generating synthetic intermediate convolutional features from label masks, which enhances the training process and improves semantic segmentation performance on challenging datasets.

Contribution

It proposes the first synthesis approach specifically designed for intermediate convolutional features to augment training data for semantic segmentation.

Findings

Improved segmentation accuracy on Cityscapes dataset

Enhanced performance on ADE20K dataset

Synthetic features effectively augment training data

Abstract

Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss. However, it remains challenging to successfully leverage synthetic data for improving semantic segmentation with additional synthetic images. Therefore, we suggest to generate intermediate convolutional features and propose the first synthesis approach that is catered to such intermediate convolutional features. This allows us to generate new features from label masks and include them successfully into the training procedure in order to improve the performance of semantic segmentation. Experimental results and analysis on two challenging datasets Cityscapes and ADE20K show that our generated feature improves performance on segmentation tasks.