RuleFlow : Generating Reusable Program Optimizations with LLMs

TL;DR

RuleFlow introduces a hybrid framework that leverages LLMs to discover program optimizations, converts them into reusable rules, and integrates them into a compiler, achieving state-of-the-art performance in Pandas program optimization.

Contribution

It presents a novel three-stage hybrid approach that decouples optimization discovery from deployment, enabling reliable and efficient program optimization with LLMs.

Findings

Achieves up to 4.3x speedup over previous SOTA Pandas optimizer

Demonstrates 1914.9x speedup over prior systems-based approaches

Sets new state-of-the-art performance on PandasBench

Abstract

Optimizing Pandas programs is a challenging problem. Existing systems and compiler-based approaches offer reliability but are either heavyweight or support only a limited set of optimizations. Conversely, using LLMs in a per-program optimization methodology can synthesize nontrivial optimizations, but is unreliable, expensive, and offers a low yield. In this work, we introduce a hybrid approach that works in a 3-stage manner that decouples discovery from deployment and connects them via a novel bridge. First, it discovers per-program optimizations (discovery). Second, they are converted into generalised rewrite rules (bridge). Finally, these rules are incorporated into a compiler that can automatically apply them wherever applicable, eliminating repeated reliance on LLMs (deployment). We demonstrate that RuleFlow is the new state-of-the-art (SOTA) Pandas optimization framework on PandasBench, a challenging Pandas benchmark consisting of Python notebooks. Across these notebooks, we achieve a speedup of up to 4.3x over Dias, the previous compiler-based SOTA, and 1914.9x over Modin, the previous systems-based SOTA. Our code is available at https://github.com/ADAPT-uiuc/RuleFlow.