Diffusion Fuzzy System: Fuzzy Rule Guided Latent Multi-Path Diffusion Modeling

TL;DR

This paper introduces a Diffusion Fuzzy System (DFS) that guides multi-path diffusion modeling with fuzzy rules, improving image feature capture, computational efficiency, and image quality in generative diffusion models.

Contribution

The paper proposes a novel fuzzy rule-guided multi-path diffusion model that enhances feature learning, reduces computational costs, and improves image generation quality.

Findings

DFS achieves more stable training and faster convergence.

DFS surpasses baseline models in image quality and text-image alignment.

DFS shows improved accuracy in generated images compared to references.

Abstract

Diffusion models have emerged as a leading technique for generating images due to their ability to create high-resolution and realistic images. Despite their strong performance, diffusion models still struggle in managing image collections with significant feature differences. They often fail to capture complex features and produce conflicting results. Research has attempted to address this issue by learning different regions of an image through multiple diffusion paths and then combining them. However, this approach leads to inefficient coordination among multiple paths and high computational costs. To tackle these issues, this paper presents a Diffusion Fuzzy System (DFS), a latent-space multi-path diffusion model guided by fuzzy rules. DFS offers several advantages. First, unlike traditional multi-path diffusion methods, DFS uses multiple diffusion paths, each dedicated to learning a specific class of image features. By assigning each path to a different feature type, DFS overcomes the limitations of multi-path models in capturing heterogeneous image features. Second, DFS employs rule-chain-based reasoning to dynamically steer the diffusion process and enable efficient coordination among multiple paths. Finally, DFS introduces a fuzzy membership-based latent-space compression mechanism to reduce the computational costs of multi-path diffusion effectively. We tested our method on three public datasets: LSUN Bedroom, LSUN Church, and MS COCO. The results show that DFS achieves more stable training and faster convergence than existing single-path and multi-path diffusion models. Additionally, DFS surpasses baseline models in both image quality and alignment between text and images, and also shows improved accuracy when comparing generated images to target references.