ObjectComposer: Consistent Generation of Multiple Objects Without Fine-tuning

TL;DR

ObjectComposer is a training-free method that enables consistent multi-object image generation from text prompts, leveraging preexisting models without fine-tuning, useful for applications like comic illustrations.

Contribution

It introduces a novel, training-free approach for generating consistent multi-object images without modifying underlying diffusion model weights.

Findings

Successfully generates multi-object compositions with consistent object appearances.

Operates without fine-tuning, reducing computational costs.

Builds upon BLIP-Diffusion for object-specific image generation.

Abstract

Recent text-to-image generative models can generate high-fidelity images from text prompts. However, these models struggle to consistently generate the same objects in different contexts with the same appearance. Consistent object generation is important to many downstream tasks like generating comic book illustrations with consistent characters and setting. Numerous approaches attempt to solve this problem by extending the vocabulary of diffusion models through fine-tuning. However, even lightweight fine-tuning approaches can be prohibitively expensive to run at scale and in real-time. We introduce a method called ObjectComposer for generating compositions of multiple objects that resemble user-specified images. Our approach is training-free, leveraging the abilities of preexisting models. We build upon the recent BLIP-Diffusion model, which can generate images of single objects specified by reference images. ObjectComposer enables the consistent generation of compositions containing multiple specific objects simultaneously, all without modifying the weights of the underlying models.