(Security) Assertions by Large Language Models

TL;DR

This paper explores using large language models to automatically generate security assertions for hardware verification, aiming to improve security verification processes with AI-driven code generation.

Contribution

It introduces an evaluation framework and benchmark suite to assess LLMs' ability to generate hardware security assertions from natural language prompts.

Findings

LLMs can generate assertions with varying accuracy based on prompt detail

The framework enables systematic evaluation of LLMs for hardware security assertion generation

Benchmark suite provides real-world hardware designs for testing assertion generation capabilities

Abstract

The security of computer systems typically relies on a hardware root of trust. As vulnerabilities in hardware can have severe implications on a system, there is a need for techniques to support security verification activities. Assertion-based verification is a popular verification technique that involves capturing design intent in a set of assertions that can be used in formal verification or testing-based checking. However, writing security-centric assertions is a challenging task. In this work, we investigate the use of emerging large language models (LLMs) for code generation in hardware assertion generation for security, where primarily natural language prompts, such as those one would see as code comments in assertion files, are used to produce SystemVerilog assertions. We focus our attention on a popular LLM and characterize its ability to write assertions out of the box, given varying levels of detail in the prompt. We design an evaluation framework that generates a variety of prompts, and we create a benchmark suite comprising real-world hardware designs and corresponding golden reference assertions that we want to generate with the LLM.