Event-Customized Image Generation

TL;DR

This paper introduces event-customized image generation, enabling the creation of complex scene images based on a single reference, by proposing a novel training-free method called FreeEvent that captures actions, poses, and interactions.

Contribution

The paper presents a new task of event-customized image generation and proposes a training-free method, FreeEvent, to accurately generate complex scenes with target entities and events.

Findings

FreeEvent effectively captures complex events in generated images.

The method outperforms baseline approaches in benchmarks.

Extensive experiments validate the approach's effectiveness.

Abstract

Customized Image Generation, generating customized images with user-specified concepts, has raised significant attention due to its creativity and novelty. With impressive progress achieved in subject customization, some pioneer works further explored the customization of action and interaction beyond entity (i.e., human, animal, and object) appearance. However, these approaches only focus on basic actions and interactions between two entities, and their effects are limited by insufficient ''exactly same'' reference images. To extend customized image generation to more complex scenes for general real-world applications, we propose a new task: event-customized image generation. Given a single reference image, we define the ''event'' as all specific actions, poses, relations, or interactions between different entities in the scene. This task aims at accurately capturing the complex event and generating customized images with various target entities. To solve this task, we proposed a novel training-free event customization method: FreeEvent. Specifically, FreeEvent introduces two extra paths alongside the general diffusion denoising process: 1) Entity switching path: it applies cross-attention guidance and regulation for target entity generation. 2) Event transferring path: it injects the spatial feature and self-attention maps from the reference image to the target image for event generation. To further facilitate this new task, we collected two evaluation benchmarks: SWiG-Event and Real-Event. Extensive experiments and ablations have demonstrated the effectiveness of FreeEvent.