Explainability-in-Action: Enabling Expressive Manipulation and Tacit Understanding by Bending Diffusion Models in ComfyUI

TL;DR

This paper introduces a craft-based explainability approach for diffusion models, enabling artists to manipulate and understand model components interactively within ComfyUI, fostering artistic engagement and control.

Contribution

It presents a novel plugin for ComfyUI that allows real-time model manipulation, promoting hands-on understanding and creative control over large generative models.

Findings

Artists can develop intuition about model components through interactive manipulation.

Model-bending enhances transparency and creative engagement with diffusion models.

The approach supports sustained artistic practice and modifiability.

Abstract

Explainable AI (XAI) in creative contexts can go beyond transparency to support artistic engagement, modifiability, and sustained practice. While curated datasets and training human-scale models can offer artists greater agency and control, large-scale generative models like text-to-image diffusion systems often obscure these possibilities. We suggest that even large models can be treated as creative materials if their internal structure is exposed and manipulable. We propose a craft-based approach to explainability rooted in long-term, hands-on engagement akin to Sch\"on's "reflection-in-action" and demonstrate its application through a model-bending and inspection plugin integrated into the node-based interface of ComfyUI. We demonstrate that by interactively manipulating different parts of a generative model, artists can develop an intuition about how each component influences the output.