Assessing GPT-4-Vision's Capabilities in UML-Based Code Generation

TL;DR

This study evaluates GPT-4-Vision's ability to convert UML class diagrams into Java code, showing high accuracy for single-class diagrams but weaker performance with multi-class diagrams, indicating potential and areas for improvement.

Contribution

First assessment of GPT-4-Vision's effectiveness in UML diagram to code translation, highlighting its strengths and limitations in automated software generation.

Findings

88% element accuracy in UML diagram code generation

High proficiency with single-class UML diagrams

Reduced performance with multi-class UML diagrams

Abstract

The emergence of advanced neural networks has opened up new ways in automated code generation from conceptual models, promising to enhance software development processes. This paper presents a preliminary evaluation of GPT-4-Vision, a state-of-the-art deep learning model, and its capabilities in transforming Unified Modeling Language (UML) class diagrams into fully operating Java class files. In our study, we used exported images of 18 class diagrams comprising 10 single-class and 8 multi-class diagrams. We used 3 different prompts for each input, and we manually evaluated the results. We created a scoring system in which we scored the occurrence of elements found in the diagram within the source code. On average, the model was able to generate source code for 88% of the elements shown in the diagrams. Our results indicate that GPT-4-Vision exhibits proficiency in handling single-class UML diagrams, successfully transforming them into syntactically correct class files. However, for multi-class UML diagrams, the model's performance is weaker compared to single-class diagrams. In summary, further investigations are necessary to exploit the model's potential completely.