Semantic Editing On Segmentation Map Via Multi-Expansion Loss

TL;DR

This paper introduces MExGAN, a novel model for semantic editing of segmentation maps that uses a multi-expansion loss to improve boundary accuracy, demonstrating competitive results in image editing tasks.

Contribution

The paper proposes a new Multi-Expansion loss and an Approximated MEx loss to enhance boundary alignment in semantic map editing, with training on full images for better performance.

Findings

MExGAN achieves improved boundary accuracy in segmentation map editing.

The model demonstrates competitive results on multiple datasets.

Training on full images enhances overall editing quality.

Abstract

Semantic editing on segmentation map has been proposed as an intermediate interface for image generation, because it provides flexible and strong assistance in various image generation tasks. This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. Even though recent studies apply global and local adversarial losses extensively to generate images for higher image quality, we find that they suffer from the misalignment of the boundary area in the mask area. To address this, we propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss implemented by adversarial losses on MEx areas. Each MEx area has the mask area of the generation as the majority and the boundary of original context as the minority. To boost convenience and stability of MEx loss, we further propose an Approximated MEx (A-MEx) loss. Besides, in contrast to previous model that builds training data for semantic editing on segmentation map with part of the whole image, which leads to model performance degradation, MExGAN applies the whole image to build the training data. Extensive experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.