FabricGen: Microstructure-Aware Woven Fabric Generation

TL;DR

FabricGen is a comprehensive framework that combines fine-tuned diffusion models and a language-guided procedural geometric model to generate highly detailed and realistic woven fabric textures from textual descriptions.

Contribution

We introduce FabricGen, integrating macro-texture synthesis with microstructure-aware weaving pattern generation driven by a specialized language model.

Findings

Produces more detailed and realistic woven fabrics than previous models.

Effectively generates diverse weaving patterns from textual prompts.

Combines macro and micro-scale modeling for high-quality fabric rendering.

Abstract

Woven fabric materials are widely used in rendering applications, yet designing realistic examples typically involves multiple stages, requiring expertise in weaving principles and texture authoring. Recent advances have explored diffusion models to streamline this process; however, pre-trained diffusion models often struggle to generate intricate yarn-level details that conform to weaving rules. To address this, we present FabricGen, an end-to-end framework for generating high-quality woven fabric materials from textual descriptions. A key insight of our method is the decomposition of macro-scale textures and micro-scale weaving patterns. To generate macro-scale textures free from microstructures, we fine-tune pre-trained diffusion models on a collected dataset of microstructure-free fabrics. As for micro-scale weaving patterns, we develop an enhanced procedural geometric model capable of synthesizing natural yarn-level geometry with yarn sliding and flyaway fibers. The procedural model is driven by a specialized large language model, WeavingLLM, which is fine-tuned on an annotated dataset of formatted weaving drafts, and prompt-tuned with domain-specific fabric expertise. Through fine-tuning and prompt tuning, WeavingLLM learns to design weaving drafts and fabric parameters from textual prompts, enabling the procedural model to produce diverse weaving patterns that stick to weaving principles. The generated macro-scale texture, along with the micro-scale geometry, can be used for fabric rendering. Consequently, our framework produces materials with significantly richer detail and realism compared to prior generative models.