Optimization of Imperative Programs in a Relational Database

TL;DR

This paper introduces Froid, a framework that transforms imperative UDFs into relational algebra for efficient, set-oriented execution in databases, significantly improving performance over traditional naive approaches.

Contribution

Froid's novel automatic transformation of UDFs into relational algebra enables cost-based optimization and compiler-like improvements without extra effort.

Findings

Performance improvements of up to multiple orders of magnitude

Effective transformation of UDFs into relational algebra

Enhanced optimization and parallel execution capabilities

Abstract

For decades, RDBMSs have supported declarative SQL as well as imperative functions and procedures as ways for users to express data processing tasks. While the evaluation of declarative SQL has received a lot of attention resulting in highly sophisticated techniques, the evaluation of imperative programs has remained naive and highly inefficient. Imperative programs offer several benefits over SQL and hence are often preferred and widely used. But unfortunately, their abysmal performance discourages, and even prohibits their use in many situations. We address this important problem that has hitherto received little attention. We present Froid, an extensible framework for optimizing imperative programs in relational databases. Froid's novel approach automatically transforms entire User Defined Functions (UDFs) into relational algebraic expressions, and embeds them into the calling SQL query. This form is now amenable to cost-based optimization and results in efficient, set-oriented, parallel plans as opposed to inefficient, iterative, serial execution of UDFs. Froid's approach additionally brings the benefits of many compiler optimizations to UDFs with no additional implementation effort. We describe the design of Froid and present our experimental evaluation that demonstrates performance improvements of up to multiple orders of magnitude on real workloads.