Large-scale Text-to-Image Generation Models for Visual Artists' Creative Works

TL;DR

This paper explores how visual artists adopt large-scale text-to-image generation models like DALL-E, highlighting their roles in automation, idea expansion, and communication facilitation, based on interviews and literature review.

Contribution

It provides a systematic study of artists' adoption of LTGMs and offers four design guidelines for future intelligent user interfaces utilizing these models.

Findings

Artists find LTGMs highly usable for automating creation processes.

LTGMs support idea exploration and expansion for artists.

Artists use LTGMs to facilitate communication and mediation.

Abstract

Large-scale Text-to-image Generation Models (LTGMs) (e.g., DALL-E), self-supervised deep learning models trained on a huge dataset, have demonstrated the capacity for generating high-quality open-domain images from multi-modal input. Although they can even produce anthropomorphized versions of objects and animals, combine irrelevant concepts in reasonable ways, and give variation to any user-provided images, we witnessed such rapid technological advancement left many visual artists disoriented in leveraging LTGMs more actively in their creative works. Our goal in this work is to understand how visual artists would adopt LTGMs to support their creative works. To this end, we conducted an interview study as well as a systematic literature review of 72 system/application papers for a thorough examination. A total of 28 visual artists covering 35 distinct visual art domains acknowledged LTGMs' versatile roles with high usability to support creative works in automating the creation process (i.e., automation), expanding their ideas (i.e., exploration), and facilitating or arbitrating in communication (i.e., mediation). We conclude by providing four design guidelines that future researchers can refer to in making intelligent user interfaces using LTGMs.