ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback

TL;DR

ComfyMind introduces a structured, feedback-driven framework for robust, flexible general-purpose AI generation, integrating semantic workflow abstraction and hierarchical planning to improve stability and performance across diverse tasks.

Contribution

It proposes a novel semantic workflow interface and search tree planning mechanism that enhance the robustness and adaptability of open-source general-purpose generative AI systems.

Findings

Outperforms existing open-source baselines on multiple benchmarks

Achieves performance comparable to GPT-Image-1

Demonstrates improved stability and flexibility in complex workflows

Abstract

With the rapid advancement of generative models, general-purpose generation has gained increasing attention as a promising approach to unify diverse tasks across modalities within a single system. Despite this progress, existing open-source frameworks often remain fragile and struggle to support complex real-world applications due to the lack of structured workflow planning and execution-level feedback. To address these limitations, we present ComfyMind, a collaborative AI system designed to enable robust and scalable general-purpose generation, built on the ComfyUI platform. ComfyMind introduces two core innovations: Semantic Workflow Interface (SWI) that abstracts low-level node graphs into callable functional modules described in natural language, enabling high-level composition and reducing structural errors; Search Tree Planning mechanism with localized feedback execution, which models generation as a hierarchical decision process and allows adaptive correction at each stage. Together, these components improve the stability and flexibility of complex generative workflows. We evaluate ComfyMind on three public benchmarks: ComfyBench, GenEval, and Reason-Edit, which span generation, editing, and reasoning tasks. Results show that ComfyMind consistently outperforms existing open-source baselines and achieves performance comparable to GPT-Image-1. ComfyMind paves a promising path for the development of open-source general-purpose generative AI systems. Project page: https://github.com/LitaoGuo/ComfyMind