Text2Arch: A Dataset for Generating Scientific Architecture Diagrams from Natural Language Descriptions

TL;DR

This paper introduces Text2Arch, a new dataset for generating scientific architecture diagrams from text, and demonstrates how fine-tuned language models can produce high-fidelity diagrams, outperforming baselines.

Contribution

The creation of a comprehensive dataset, Text2Arch, and the demonstration of effective fine-tuning of language models for diagram generation from text descriptions.

Findings

Models trained on Text2Arch outperform baseline models like DiagramAgent.

Fine-tuned models achieve performance comparable to GPT-4o in in-context learning.

The dataset and models are publicly available for research use.

Abstract

Communicating complex system designs or scientific processes through text alone is inefficient and prone to ambiguity. A system that automatically generates scientific architecture diagrams from text with high semantic fidelity can be useful in multiple applications like enterprise architecture visualization, AI-driven software design, and educational content creation. Hence, in this paper, we focus on leveraging language models to perform semantic understanding of the input text description to generate intermediate code that can be processed to generate high-fidelity architecture diagrams. Unfortunately, no clean large-scale open-access dataset exists, implying lack of any effective open models for this task. Hence, we contribute a comprehensive dataset, \system, comprising scientific architecture images, their corresponding textual descriptions, and associated DOT code representations. Leveraging this resource, we fine-tune a suite of small language models, and also perform in-context learning using GPT-4o. Through extensive experimentation, we show that \system{} models significantly outperform existing baseline models like DiagramAgent and perform at par with in-context learning-based generations from GPT-4o. We make the code, data and models publicly available.