Spatial Transformation for Image Composition via Correspondence Learning

TL;DR

This paper introduces a new dataset and a correspondence learning network to improve geometric consistency in image composition, significantly enhancing the quality of composite images.

Contribution

The work presents a novel dataset for geometric correction and a correspondence learning network that models foreground-background relations for better image warping.

Findings

CorrelNet outperforms previous methods on the STRAT dataset.

The dataset enables more reliable evaluation of geometric correction methods.

Filtering noisy coordinate pairs improves warping accuracy.

Abstract

When using cut-and-paste to acquire a composite image, the geometry inconsistency between foreground and background may severely harm its fidelity. To address the geometry inconsistency in composite images, several existing works learned to warp the foreground object for geometric correction. However, the absence of annotated dataset results in unsatisfactory performance and unreliable evaluation. In this work, we contribute a Spatial TRAnsformation for virtual Try-on (STRAT) dataset covering three typical application scenarios. Moreover, previous works simply concatenate foreground and background as input without considering their mutual correspondence. Instead, we propose a novel correspondence learning network (CorrelNet) to model the correspondence between foreground and background using cross-attention maps, based on which we can predict the target coordinate that each source coordinate of foreground should be mapped to on the background. Then, the warping parameters of foreground object can be derived from pairs of source and target coordinates. Additionally, we learn a filtering mask to eliminate noisy pairs of coordinates to estimate more accurate warping parameters. Extensive experiments on our STRAT dataset demonstrate that our proposed CorrelNet performs more favorably against previous methods.