Learning Graph Representation of Agent Diffusers

TL;DR

This paper introduces LGR-AD, a multi-agent system that models diffusion-based image generation as a dynamic, graph-structured collaboration to improve adaptability and performance in complex visual tasks.

Contribution

The paper proposes a novel multi-agent framework with graph neural networks and a coordination mechanism to enhance diffusion models' flexibility and effectiveness.

Findings

LGR-AD outperforms traditional diffusion models on multiple benchmarks.

The multi-agent approach improves adaptability in dynamic image generation.

Theoretical analysis supports the effectiveness of the proposed system.

Abstract

Diffusion-based generative models have significantly advanced text-to-image synthesis, demonstrating impressive text comprehension and zero-shot generalization. These models refine images from random noise based on textual prompts, with initial reliance on text input shifting towards enhanced visual fidelity over time. This transition suggests that static model parameters might not optimally address the distinct phases of generation. We introduce LGR-AD (Learning Graph Representation of Agent Diffusers), a novel multi-agent system designed to improve adaptability in dynamic computer vision tasks. LGR-AD models the generation process as a distributed system of interacting agents, each representing an expert sub-model. These agents dynamically adapt to varying conditions and collaborate through a graph neural network that encodes their relationships and performance metrics. Our approach employs a coordination mechanism based on top-$k$ maximum spanning trees, optimizing the generation process. Each agent's decision-making is guided by a meta-model that minimizes a novel loss function, balancing accuracy and diversity. Theoretical analysis and extensive empirical evaluations show that LGR-AD outperforms traditional diffusion models across various benchmarks, highlighting its potential for scalable and flexible solutions in complex image generation tasks. Code is available at: https://github.com/YousIA/LGR_AD