AutoFSM: A Multi-agent Framework for FSM Code Generation with IR and SystemC-Based Testing

TL;DR

AutoFSM is a multi-agent framework that enhances FSM code generation from large language models by using a structured IR, SystemC-based testing, and an innovative benchmark, significantly improving accuracy and debugging efficiency.

Contribution

This paper introduces AutoFSM, the first framework integrating IR and SystemC-based testing for FSM code generation, addressing syntax errors and debugging challenges in LLM-based hardware design.

Findings

Up to 11.94% improvement in pass rate over baseline

Up to 17.62% reduction in syntax error rate

First hierarchical FSM benchmark in the field

Abstract

With the rapid advancement of large language models (LLMs) in code generation, their applications in hardware design are receiving growing attention. However, existing LLMs face several challenges when generating Verilog code for finite state machine (FSM) control logic, including frequent syntax errors, low debugging efficiency, and heavy reliance on test benchmarks. To address these challenges, this paper proposes AutoFSM, a multi-agent collaborative framework designed for FSM code generation tasks. AutoFSM introduces a structurally clear intermediate representation (IR) to reduce syntax error rate during code generation and provides a supporting toolchain to enable automatic translation from IR to Verilog. Furthermore, AutoFSM is the first to integrate SystemC-based modeling with automatic testbench generation, thereby improving debugging efficiency and feedback quality. To systematically evaluate the framework's performance, we construct SKT-FSM, the first hierarchical FSM benchmark in the field, comprising 67 FSM samples across different complexity levels. Experimental results show that, under the same base LLM, AutoFSM consistently outperforms the open-source framework MAGE on the SKT-FSM benchmark, achieving up to an 11.94% improvement in pass rate and up to a 17.62% reduction in syntax error rate. These results demonstrate the potential of combining LLMs with structured IR and automated testing to improve the reliability and scalability of register-transfer level (RTL) code generation.