A Framework for Critical Evaluation of Text-to-Image Models: Integrating Art Historical Analysis, Artistic Exploration, and Critical Prompt Engineering

TL;DR

This paper introduces an interdisciplinary framework combining art historical analysis, artistic exploration, and prompt engineering to critically evaluate text-to-image models, revealing biases and societal implications beyond traditional metrics.

Contribution

It presents a novel comprehensive approach that integrates art analysis and creative prompts to assess model capabilities and biases more effectively.

Findings

Revealed gender, race, and cultural biases in models

Uncovered hidden potentials through artistic exploration

Demonstrated practical application via case studies

Abstract

This paper proposes a novel interdisciplinary framework for the critical evaluation of text-to-image models, addressing the limitations of current technical metrics and bias studies. By integrating art historical analysis, artistic exploration, and critical prompt engineering, the framework offers a more nuanced understanding of these models' capabilities and societal implications. Art historical analysis provides a structured approach to examine visual and symbolic elements, revealing potential biases and misrepresentations. Artistic exploration, through creative experimentation, uncovers hidden potentials and limitations, prompting critical reflection on the algorithms' assumptions. Critical prompt engineering actively challenges the model's assumptions, exposing embedded biases. Case studies demonstrate the framework's practical application, showcasing how it can reveal biases related to gender, race, and cultural representation. This comprehensive approach not only enhances the evaluation of text-to-image models but also contributes to the development of more equitable, responsible, and culturally aware AI systems.