Code Detection for Hardware Acceleration Using Large Language Models

TL;DR

This paper explores the use of large language models for code detection in hardware acceleration, introducing a novel prompting strategy that significantly improves accuracy over conventional methods.

Contribution

It is the first to analyze code detection with LLMs and proposes a new prompting approach that reduces false positives and enhances detection accuracy.

Findings

Conventional prompts yield high precision but low accuracy due to false positives.

The novel prompting strategy greatly improves overall accuracy for key kernels.

Results challenge existing code detection methods with superior performance.

Abstract

Large language models (LLMs) have been massively applied to many tasks, often surpassing state-of-the-art approaches. While their effectiveness in code generation has been extensively studied (e.g., AlphaCode), their potential for code detection remains unexplored. This work presents the first analysis of code detection using LLMs. Our study examines essential kernels, including matrix multiplication, convolution, and fast-fourier transform, implemented in C/C++. We propose both a preliminary, naive prompt and a novel prompting strategy for code detection. Results reveal that conventional prompting achieves great precision but poor accuracy (68.8%, 22.3%, and 79.2% for GEMM, convolution, and FFT, respectively) due to a high number of false positives. Our novel prompting strategy substantially reduces false positives, resulting in excellent overall accuracy (91.1%, 97.9%, and 99.7%, respectively). These results pose a considerable challenge to existing state-of-the-art code detection methods.