VecIntrinBench: Benchmarking Cross-Architecture Intrinsic Code Migration for RISC-V Vector

TL;DR

This paper introduces VecIntrinBench, a comprehensive benchmark for evaluating intrinsic code migration to RISC-V Vector architecture, including 50 tasks and analysis of code migration approaches like LLMs and rule-based methods.

Contribution

It presents the first benchmark for RVV intrinsic migration, encompassing diverse tasks and enabling systematic evaluation of migration techniques.

Findings

LLMs perform comparably to rule-based methods for RISC-V code migration.

LLMs deliver superior performance in intrinsic code migration.

The benchmark is open-sourced for community use.

Abstract

Intrinsic functions are specialized functions provided by the compiler that efficiently operate on architecture-specific hardware, allowing programmers to write optimized code in a high-level language that fully exploits hardware features. Using intrinsics to vectorize core code blocks is a standard optimization method in high-performance libraries, often requiring specific vector optimization implementations for multiple mainstream architectures. The promising RISC-V software ecosystem has a significant demand for algorithm library migration and adaptation. Translating existing intrinsic functions to RISC-V Vector (RVV) intrinsic functions across architectures is currently a mainstream approach. Rule-based intrinsic mapping methods and LLM-based code generation can help developers address the code migration challenge. However, existing intrinsic code benchmarks focus on mainstream SIMD intrinsics and lack support for the emerging RISC-V architecture. There is currently no benchmark that comprehensively evaluates the intrinsic migration capabilities for the RVV extension. To fill this gap, we propose VecIntrinBench, the first intrinsic benchmark encompassing RVV extensions. It includes 50 function-level tasks from open source repositories, implemented as scalars, RVV intrinsics, Arm Neon intrinsics, and x86 intrinsics, along with comprehensive functional and performance test cases. We systematically evaluated various code migration approaches on VecIntrinBench, yielding a series of insightful findings. The results demonstrate that advanced Large Language Models (LLMs) achieve a similar effect as rule-based mapping approaches for RISC-V code migration, while also delivering superior performance. We further analyze the reasons and identify future directions for LLM development in the code migration field. The VecIntrinBench is open-sourced to benefit the broader community and developers.