Illustrating Classic Brazilian Books using a Text-To-Image Diffusion Model

TL;DR

This paper explores using a text-to-image diffusion model to generate illustrations for classic Brazilian books, assessing its practicality, strengths, and limitations in enriching literary experiences.

Contribution

It demonstrates the feasibility of applying Stable Diffusion LDM to illustrate literary works, highlighting both its potential and challenges in this novel application.

Findings

Generated images are contextually relevant and distinctive.

The model captures many literary details but sometimes misses nuanced depictions.

AI illustrations can enhance reader engagement but face limitations in accuracy.

Abstract

In recent years, Generative Artificial Intelligence (GenAI) has undergone a profound transformation in addressing intricate tasks involving diverse modalities such as textual, auditory, visual, and pictorial generation. Within this spectrum, text-to-image (TTI) models have emerged as a formidable approach to generating varied and aesthetically appealing compositions, spanning applications from artistic creation to realistic facial synthesis, and demonstrating significant advancements in computer vision, image processing, and multimodal tasks. The advent of Latent Diffusion Models (LDMs) signifies a paradigm shift in the domain of AI capabilities. This article delves into the feasibility of employing the Stable Diffusion LDM to illustrate literary works. For this exploration, seven classic Brazilian books have been selected as case studies. The objective is to ascertain the practicality of this endeavor and to evaluate the potential of Stable Diffusion in producing illustrations that augment and enrich the reader's experience. We will outline the beneficial aspects, such as the capacity to generate distinctive and contextually pertinent images, as well as the drawbacks, including any shortcomings in faithfully capturing the essence of intricate literary depictions. Through this study, we aim to provide a comprehensive assessment of the viability and efficacy of utilizing AI-generated illustrations in literary contexts, elucidating both the prospects and challenges encountered in this pioneering application of technology.