Painterly Image Harmonization in Dual Domains

TL;DR

This paper introduces a novel dual-domain network for painterly image harmonization, effectively adjusting composite images with photographic foregrounds and painterly backgrounds by leveraging spatial and frequency domain techniques.

Contribution

It proposes a dual-domain generator and discriminator with AdaIN and ResFFT modules, advancing harmonization quality in both spatial and frequency domains.

Findings

Outperforms existing methods on benchmark datasets

Effective in both spatial and frequency domain harmonization

Generates visually harmonious composite images

Abstract

Image harmonization aims to produce visually harmonious composite images by adjusting the foreground appearance to be compatible with the background. When the composite image has photographic foreground and painterly background, the task is called painterly image harmonization. There are only few works on this task, which are either time-consuming or weak in generating well-harmonized results. In this work, we propose a novel painterly harmonization network consisting of a dual-domain generator and a dual-domain discriminator, which harmonizes the composite image in both spatial domain and frequency domain. The dual-domain generator performs harmonization by using AdaIN modules in the spatial domain and our proposed ResFFT modules in the frequency domain. The dual-domain discriminator attempts to distinguish the inharmonious patches based on the spatial feature and frequency feature of each patch, which can enhance the ability of generator in an adversarial manner. Extensive experiments on the benchmark dataset show the effectiveness of our method. Our code and model are available at https://github.com/bcmi/PHDNet-Painterly-Image-Harmonization.