Foreground-aware Semantic Representations for Image Harmonization

TL;DR

This paper introduces a novel image harmonization method that leverages pre-trained high-level feature spaces, significantly improving visual consistency in composite images and setting new benchmarks in MSE and PSNR metrics.

Contribution

It proposes a new architecture combining encoder-decoder models with pre-trained foreground-aware networks for better image harmonization.

Findings

Achieved state-of-the-art results on image harmonization benchmarks.

Outperformed previous methods in MSE and PSNR metrics.

Demonstrated the effectiveness of using pre-trained high-level features.

Abstract

Image harmonization is an important step in photo editing to achieve visual consistency in composite images by adjusting the appearances of foreground to make it compatible with background. Previous approaches to harmonize composites are based on training of encoder-decoder networks from scratch, which makes it challenging for a neural network to learn a high-level representation of objects. We propose a novel architecture to utilize the space of high-level features learned by a pre-trained classification network. We create our models as a combination of existing encoder-decoder architectures and a pre-trained foreground-aware deep high-resolution network. We extensively evaluate the proposed method on existing image harmonization benchmark and set up a new state-of-the-art in terms of MSE and PSNR metrics. The code and trained models are available at \url{https://github.com/saic-vul/image_harmonization}.