GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing

TL;DR

GALA introduces a novel approach for object search in image compositing that considers geometry and lighting compatibility, achieving state-of-the-art results and handling diverse scenarios including user-provided backgrounds without bounding boxes.

Contribution

This work presents GALA, a new method that models geometry and lighting factors for improved object search in compositing, surpassing previous compatibility-focused approaches.

Findings

Achieves state-of-the-art results on CAIS dataset

Generalizes well to large-scale open-world datasets

Effectively handles non-box scenarios with background-only inputs

Abstract

Compositing-aware object search aims to find the most compatible objects for compositing given a background image and a query bounding box. Previous works focus on learning compatibility between the foreground object and background, but fail to learn other important factors from large-scale data, i.e. geometry and lighting. To move a step further, this paper proposes GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open-world image compositing. Remarkably, it achieves state-of-the-art results on the CAIS dataset and generalizes well on large-scale open-world datasets, i.e. Pixabay and Open Images. In addition, our method can effectively handle non-box scenarios, where users only provide background images without any input bounding box. A web demo (see supplementary materials) is built to showcase applications of the proposed method for compositing-aware search and automatic location/scale prediction for the foreground object.