CareCom: Generative Image Composition with Calibrated Reference Features

TL;DR

This paper introduces CareCom, a generative image composition model that uses multiple calibrated reference images to improve detail preservation and pose/view adjustment in inserting foreground objects into backgrounds.

Contribution

The paper extends existing models to handle multiple references and proposes a calibration method for reference features to enhance composition quality.

Findings

Significant improvement in detail preservation and pose adjustment.

Effective use of multiple reference images for better composition.

Validated on MVImgNet and MureCom datasets.

Abstract

Image composition aims to seamlessly insert foreground object into background. Despite the huge progress in generative image composition, the existing methods are still struggling with simultaneous detail preservation and foreground pose/view adjustment. To address this issue, we extend the existing generative composition model to multi-reference version, which allows using arbitrary number of foreground reference images. Furthermore, we propose to calibrate the global and local features of foreground reference images to make them compatible with the background information. The calibrated reference features can supplement the original reference features with useful global and local information of proper pose/view. Extensive experiments on MVImgNet and MureCom demonstrate that the generative model can greatly benefit from the calibrated reference features.