From Image to UML: First Results of Image Based UML Diagram Generation Using LLMs
Aaron Conrardy, Jordi Cabot

TL;DR
This paper investigates using large language models to automatically convert hand-drawn UML diagrams into formal models, aiming to streamline the transition from informal sketches to machine-readable representations in software engineering.
Contribution
It presents an initial exploration of LLMs for image-based UML diagram generation, demonstrating potential and identifying current limitations for future improvements.
Findings
LLMs can generate UML models from images with promising accuracy
Current limitations require human oversight to ensure correctness
Approach can be integrated into model-driven engineering pipelines
Abstract
In software engineering processes, systems are first specified using a modeling language such as UML. These initial designs are often collaboratively created, many times in meetings where different domain experts use whiteboards, paper or other types of quick supports to create drawings and blueprints that then will need to be formalized. These proper, machine-readable, models are key to ensure models can be part of automated processes (e.g. input of a low-code generation pipeline, a model-based testing system, ...). But going from hand-drawn diagrams to actual models is a time-consuming process that sometimes ends up with such drawings just added as informal images to the software documentation, reducing their value a lot. To avoid this tedious task, we explore the usage of Large Language Models (LLM) to generate the formal representation of (UML) models from a given drawing. More…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModel-Driven Software Engineering Techniques · Semantic Web and Ontologies · Data Visualization and Analytics
