Designing FSMs Specifications from Requirements with GPT 4.0

TL;DR

This paper introduces an LLM-based framework utilizing GPT-4.0 to automatically generate and repair finite state machines from natural language requirements, enhancing system testing and reliability.

Contribution

It proposes a novel LLM-driven approach for designing and repairing FSMs from textual requirements, with experimental validation and expert-centric mutation techniques.

Findings

LLM can effectively generate FSMs from natural language requirements.

The framework improves FSM quality through mutation and test generation.

Experimental results demonstrate the potential of LLMs in MDE applications.

Abstract

Finite state machines (FSM) are executable formal specifications of reactive systems. These machines are designed based on systems' requirements. The requirements are often recorded in textual documents written in natural languages. FSMs play a crucial role in different phases of the model-driven system engineering (MDE). For example, they serve to automate testing activities. FSM quality is critical: the lower the quality of FSM, the higher the number of faults surviving the testing phase and the higher the risk of failure of the systems in production, which could lead to catastrophic scenarios. Therefore, this paper leverages recent advances in the domain of LLM to propose an LLM-based framework for designing FSMs from requirements. The framework also suggests an expert-centric approach based on FSM mutation and test generation for repairing the FSMs produced by LLMs. This paper also provides an experimental analysis and evaluation of LLM's capacities in performing the tasks presented in the framework and FSM repair via various methods. The paper presents experimental results with simulated data. These results and methods bring a new analysis and vision of LLMs that are useful for further development of machine learning technology and its applications to MDE.