Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion

TL;DR

Diffusion Prism is a training-free framework that enhances diversity and morphological consistency in mask-to-image diffusion, especially for low-entropy inputs, by transforming binary masks into realistic, diverse images while preserving features.

Contribution

It introduces a novel, training-free method that improves diversity and morphology preservation in mask-to-image diffusion, applicable to biological patterns and data augmentation.

Findings

Small artificial noise improves image-denoising process.

Diffusion Prism produces diverse, realistic images from binary masks.

Method extends to various biological pattern applications.

Abstract

The emergence of generative AI and controllable diffusion has made image-to-image synthesis increasingly practical and efficient. However, when input images exhibit low entropy and sparse, the inherent characteristics of diffusion models often result in limited diversity. This constraint significantly interferes with data augmentation. To address this, we propose Diffusion Prism, a training-free framework that efficiently transforms binary masks into realistic and diverse samples while preserving morphological features. We explored that a small amount of artificial noise will significantly assist the image-denoising process. To prove this novel mask-to-image concept, we use nano-dendritic patterns as an example to demonstrate the merit of our method compared to existing controllable diffusion models. Furthermore, we extend the proposed framework to other biological patterns, highlighting its potential applications across various fields.