On the Utility of Domain Modeling Assistance with Large Language Models

TL;DR

This paper explores using large language models with few-shot learning to assist software engineers in domain modeling, aiming to improve efficiency and support despite limited domain data.

Contribution

It introduces MAGDA, a novel LLM-based tool for domain modeling, and evaluates its usability and effectiveness through a user study.

Findings

MAGDA provides valuable modeling recommendations.

The approach reduces the need for extensive domain-specific training.

User study shows positive usability and practical benefits.

Abstract

Model-driven engineering (MDE) simplifies software development through abstraction, yet challenges such as time constraints, incomplete domain understanding, and adherence to syntactic constraints hinder the design process. This paper presents a study to evaluate the usefulness of a novel approach utilizing large language models (LLMs) and few-shot prompt learning to assist in domain modeling. The aim of this approach is to overcome the need for extensive training of AI-based completion models on scarce domain-specific datasets and to offer versatile support for various modeling activities, providing valuable recommendations to software modelers. To support this approach, we developed MAGDA, a user-friendly tool, through which we conduct a user study and assess the real-world applicability of our approach in the context of domain modeling, offering valuable insights into its usability and effectiveness.