Deep Image Compositing

TL;DR

This paper introduces an end-to-end deep learning method for automatic high-quality image compositing that leverages multi-scale fusion and a self-taught training strategy, outperforming existing approaches.

Contribution

It presents a novel multi-stream fusion network and a self-taught training strategy for automatic image compositing, reducing manual editing efforts.

Findings

Outperforms existing methods qualitatively

Achieves higher quantitative compositing quality

Automatically generates high-quality composites

Abstract

Image compositing is a task of combining regions from different images to compose a new image. A common use case is background replacement of portrait images. To obtain high quality composites, professionals typically manually perform multiple editing steps such as segmentation, matting and foreground color decontamination, which is very time consuming even with sophisticated photo editing tools. In this paper, we propose a new method which can automatically generate high-quality image compositing without any user input. Our method can be trained end-to-end to optimize exploitation of contextual and color information of both foreground and background images, where the compositing quality is considered in the optimization. Specifically, inspired by Laplacian pyramid blending, a dense-connected multi-stream fusion network is proposed to effectively fuse the information from the foreground and background images at different scales. In addition, we introduce a self-taught strategy to progressively train from easy to complex cases to mitigate the lack of training data. Experiments show that the proposed method can automatically generate high-quality composites and outperforms existing methods both qualitatively and quantitatively.