Structural Mutation Based Differential Testing for FPGA Logic Synthesis Compilers

TL;DR

This paper introduces LSC-Fuzz, a Bayesian optimization-guided mutation testing approach for FPGA logic synthesis compilers, significantly improving bug detection in safety-critical FPGA applications.

Contribution

It presents a novel guided mutation strategy using Bayesian optimization for more effective FPGA compiler testing, outperforming previous simple mutation methods.

Findings

Discovered 16 bugs in FPGA logic synthesis compilers

12 bugs confirmed by official support

Enhanced bug detection with Bayesian-guided test generation

Abstract

Field Programmable Gate Arrays (FPGAs) play a crucial role in Electronic Design Automation (EDA) applications, which have been widely used in safety-critical environments, including aerospace, chip manufacturing, and medical devices. A critical step in FPGA development is logic synthesis, which enables developers to translate their software designs into hardware net lists, which facilitates the physical implementation of the chip, detailed timing and power analysis, gate-level simulation, test vector generation, and optimization and consistency checking. However, bugs or incorrect implementations in FPGA logic synthesis compilers may lead to unexpected behaviors in target wapplications, posing security risks. Therefore, it is crucial to eliminate such bugs in FPGA logic synthesis compilers. The effectiveness of existing works is still limited by its simple, blind mutation strategy. To address this challenge, we propose a guided mutation strategy based on Bayesian optimization called LSC-Fuzz to detect bugs in FPGA logic synthesis compilers. Specifically, LSC-Fuzz consists of three components: the test-program generation component, the Bayesian diversity selection component, and the equivalent check component. By performing test-program generation and Bayesian diversity selection, LSC-Fuzz generates diverse and complex HDL code, thoroughly testing the FPGA logic synthesis compilers using equivalent check to detect bugs. Through three months, LSC-Fuzz has found 16 bugs, 12 of these has been confirmed by official technical support.