Context-Consistent Semantic Image Editing with Style-Preserved Modulation

TL;DR

This paper introduces a style-preserved modulation technique for semantic image editing that maintains image-specific styles and improves boundary quality between edited and original regions.

Contribution

It proposes a novel style-preserved modulation method with a two-step process to better integrate semantic layouts while preserving context styles.

Findings

Achieves more consistent and natural editing results.

Reduces boundary artifacts between edited and original regions.

Outperforms existing methods in qualitative and quantitative evaluations.

Abstract

Semantic image editing utilizes local semantic label maps to generate the desired content in the edited region. A recent work borrows SPADE block to achieve semantic image editing. However, it cannot produce pleasing results due to style discrepancy between the edited region and surrounding pixels. We attribute this to the fact that SPADE only uses an image-independent local semantic layout but ignores the image-specific styles included in the known pixels. To address this issue, we propose a style-preserved modulation (SPM) comprising two modulations processes: The first modulation incorporates the contextual style and semantic layout, and then generates two fused modulation parameters. The second modulation employs the fused parameters to modulate feature maps. By using such two modulations, SPM can inject the given semantic layout while preserving the image-specific context style. Moreover, we design a progressive architecture for generating the edited content in a coarse-to-fine manner. The proposed method can obtain context-consistent results and significantly alleviate the unpleasant boundary between the generated regions and the known pixels.