LLM-Vectorizer: LLM-based Verified Loop Vectorizer

TL;DR

This paper explores using large-language models to automatically generate and verify vectorized code from scalar programs, achieving significant speedups and improving compiler vectorization success rates.

Contribution

It introduces a novel LLM-based approach with test feedback for generating vectorized code and applies verification techniques to ensure correctness.

Findings

Achieves 1.1x to 9.4x speedup over traditional compilers.

Verifies 38.2% of vectorizations as correct on benchmark datasets.

Demonstrates the potential of LLMs in automated code optimization.

Abstract

Vectorization is a powerful optimization technique that significantly boosts the performance of high performance computing applications operating on large data arrays. Despite decades of research on auto-vectorization, compilers frequently miss opportunities to vectorize code. On the other hand, writing vectorized code manually using compiler intrinsics is still a complex, error-prone task that demands deep knowledge of specific architecture and compilers. In this paper, we evaluate the potential of large-language models (LLMs) to generate vectorized (Single Instruction Multiple Data) code from scalar programs that process individual array elements. We propose a novel finite-state machine multi-agents based approach that harnesses LLMs and test-based feedback to generate vectorized code. Our findings indicate that LLMs are capable of producing high performance vectorized code with run-time speedup ranging from 1.1x to 9.4x as compared to the state-of-the-art compilers such as Intel Compiler, GCC, and Clang. To verify the correctness of vectorized code, we use Alive2, a leading bounded translation validation tool for LLVM IR. We describe a few domain-specific techniques to improve the scalability of Alive2 on our benchmark dataset. Overall, our approach is able to verify 38.2% of vectorizations as correct on the TSVC benchmark dataset.