SLDB: An End-To-End Heterogeneous System-on-Chip Benchmark Suite for LLM-Aided Design

TL;DR

This paper introduces SLDB, a comprehensive benchmark suite for evaluating LLMs in system-level SoC design, addressing the lack of high-level, complex design datasets in existing EDA benchmarks.

Contribution

The paper presents SLDB, a novel system-level SoC benchmark suite with diverse configurations and integration tasks, enabling better evaluation of LLMs in hardware design automation.

Findings

SLDB includes 10 baseline SoC designs with exponential configurability.

Provides full system configurations, integration code, and testing applications.

Facilitates evaluation of LLMs in complex, system-level hardware design tasks.

Abstract

Over the last few years, Large Language Models (LLMs) have emerged as a valuable tool for Electronic Design Automation (EDA). State-of-the-art research in LLM-aided design has demonstrated the ability of LLMs to generate syntactically correct RTL code, showcasing encouraging prospects for integrating AI into the hardware design process. A key enabler of these advancements is the availability of high-quality benchmarks to evaluate new approaches. However, existing datasets and benchmarks fall short of system-level design, as they focus primarily on component-level information and low-complexity designs. To address this gap, we introduce the System-Level Design Benchmark (SLDB), a dataset tailored for evaluating LLMs in system-level integration and configuration tasks. SLDB includes a curated benchmark suite of 10 baseline SoC designs, whose components can be combined into an exponential number of distinct tile-based SoCs through a synthetic library. The dataset provides full SoC configurations, accelerator integration code, communication parameters, and accelerator-aware system configurations, along with testing-application code, compatible with the ESP platform[1].