A Problem-Oriented Perspective and Anchor Verification for Code Optimization

TL;DR

This paper explores a new problem-oriented approach and an anchor verification framework to improve code optimization using LLMs, addressing local vs. global improvements and the challenge of optimization trade-offs.

Contribution

It introduces a problem-oriented perspective and anchor verification method, enabling global algorithmic improvements and reducing optimization trade-offs in code optimization with LLMs.

Findings

Enhanced correct optimization ratio

Increased speedup in code optimization

Effective mitigation of optimization tax

Abstract

Large Language Models (LLMs) have shown remarkable capabilities in solving various programming tasks, such as code generation. However, their potential for code optimization, particularly in performance enhancement, remains largely unexplored. This paper investigates the capabilities of LLMs in optimizing code for minimal execution time, addressing a critical gap in current research. The recently proposed code optimization methods construct program optimization pairs based on iterative submissions from the same programmer for the same problem. However, this approach confines LLMs to local performance improvements, neglecting global algorithmic innovation. To overcome this limitation, we adopt a completely different perspective by reconstructing the optimization pairs into a problem-oriented approach. This allows for the integration of various ideas from multiple programmers tackling the same problem. Furthermore, we observe that code optimization presents greater challenges compared to code generation, often accompanied by "optimization tax". Recognizing the inherent trade-offs in correctness and efficiency, we introduce a novel anchor verification framework to mitigate this "optimization tax". Ultimately, the problem oriented perspective combined with the anchor verification framework significantly enhances both the correct optimization ratio and speedup to new levels.