ORACLE: Leveraging Mutual Information for Consistent Character Generation with LoRAs in Diffusion Models

TL;DR

This paper presents a novel framework that leverages mutual information to improve the consistency of character generation in diffusion models, enabling more uniform visual identities across diverse prompts and settings.

Contribution

It introduces a new method that enhances character consistency in diffusion-based text-to-image models using mutual information, addressing a key challenge in creative AI applications.

Findings

Outperforms existing methods in maintaining character consistency

Demonstrates effectiveness through quantitative and qualitative analyses

Enhances the practical utility of diffusion models in creative industries

Abstract

Text-to-image diffusion models have recently taken center stage as pivotal tools in promoting visual creativity across an array of domains such as comic book artistry, children's literature, game development, and web design. These models harness the power of artificial intelligence to convert textual descriptions into vivid images, thereby enabling artists and creators to bring their imaginative concepts to life with unprecedented ease. However, one of the significant hurdles that persist is the challenge of maintaining consistency in character generation across diverse contexts. Variations in textual prompts, even if minor, can yield vastly different visual outputs, posing a considerable problem in projects that require a uniform representation of characters throughout. In this paper, we introduce a novel framework designed to produce consistent character representations from a single text prompt across diverse settings. Through both quantitative and qualitative analyses, we demonstrate that our framework outperforms existing methods in generating characters with consistent visual identities, underscoring its potential to transform creative industries. By addressing the critical challenge of character consistency, we not only enhance the practical utility of these models but also broaden the horizons for artistic and creative expression.