AMICO: Amodal Instance Composition

TL;DR

This paper introduces Amodal Instance Composition, a method for seamlessly blending imperfectly segmented objects into images by predicting their shapes and contents, improving compositing in unconstrained scenarios.

Contribution

It proposes a novel neural composition framework that handles incomplete and coarse object segments by predicting amodal contents and blending masks, advancing image composition techniques.

Findings

Achieves state-of-the-art results on COCOA and KINS benchmarks.

Demonstrates effective object insertion and de-occlusion applications.

Handles imperfect object segments better than previous methods.

Abstract

Image composition aims to blend multiple objects to form a harmonized image. Existing approaches often assume precisely segmented and intact objects. Such assumptions, however, are hard to satisfy in unconstrained scenarios. We present Amodal Instance Composition for compositing imperfect -- potentially incomplete and/or coarsely segmented -- objects onto a target image. We first develop object shape prediction and content completion modules to synthesize the amodal contents. We then propose a neural composition model to blend the objects seamlessly. Our primary technical novelty lies in using separate foreground/background representations and blending mask prediction to alleviate segmentation errors. Our results show state-of-the-art performance on public COCOA and KINS benchmarks and attain favorable visual results across diverse scenes. We demonstrate various image composition applications such as object insertion and de-occlusion.