A Novel HDL Code Generator for Effectively Testing FPGA Logic Synthesis Compilers

TL;DR

This paper introduces LegoHDL, a new HDL code generator that produces complex, diverse, and syntax-valid HDL code to effectively test FPGA logic synthesis compilers, uncovering numerous previously unknown defects.

Contribution

LegoHDL employs syntax tree guidance and extensive function libraries to generate more effective HDL test cases, surpassing existing methods like Verismith in diversity and defect detection.

Findings

LegoHDL generates more diverse HDL code than Verismith.

LegoHDL uncovers 20 new defects in FPGA compilers.

16 defects have been confirmed as critical issues.

Abstract

Field Programmable Gate Array (FPGA) logic synthesis compilers (e.g., Vivado, Iverilog, Yosys, and Quartus) are widely applied in Electronic Design Automation (EDA), such as the development of FPGA programs.However, defects (i.e., incorrect synthesis) in logic synthesis compilers may lead to unexpected behaviors in target applications, posing security risks. Therefore, it is crucial to thoroughly test logic synthesis compilers to eliminate such defects.Despite several Hardware Design Language (HDL) code generators (e.g., Verismith) have been proposed to find defects in logic synthesis compilers, the effectiveness of these generators is still limited by the simple code generation strategy and the monogeneity of the generated HDL code.This paper proposes LegoHDL, a novel method to generate syntax valid HDL code for comprehensively testing FPGA logic synthesis compilers.LegoHDL can generate more complex and diverse defect-trigger HDL code (e.g., Verilog, VHDL, and SystemVerilog) by leveraging the guidance of abstract syntax tree and the extensive function block libraries of cyber-physical systems. Extensive experiments show that the diversity and defect-trigger capability of HDL code generated by LegoHDL are significantly better than the state-of-the-art method (i.e., Verismith).In three months, LegoHDL has reported 20 new defects--many of which are deep and important; 16 of them have been confirmed.