Images Speak Volumes: User-Centric Assessment of Image Generation for Accessible Communication

TL;DR

This study evaluates the potential of text-to-image models to generate customizable, accessible images for explanatory texts, highlighting current limitations and the need for human oversight to meet user needs.

Contribution

It benchmarks multiple image generation models and assesses their suitability for creating accessible images, providing insights into their performance and user preferences.

Findings

Some models perform remarkably well

None are ready for large-scale autonomous use

Human supervision remains essential

Abstract

Explanatory images play a pivotal role in accessible and easy-to-read (E2R) texts. However, the images available in online databases are not tailored toward the respective texts, and the creation of customized images is expensive. In this large-scale study, we investigated whether text-to-image generation models can close this gap by providing customizable images quickly and easily. We benchmarked seven, four open- and three closed-source, image generation models and provide an extensive evaluation of the resulting images. In addition, we performed a user study with people from the E2R target group to examine whether the images met their requirements. We find that some of the models show remarkable performance, but none of the models are ready to be used at a larger scale without human supervision. Our research is an important step toward facilitating the creation of accessible information for E2R creators and tailoring accessible images to the target group's needs.