COMPARE: Accelerating Groupwise Comparison in Relational Databases for Data Analytics

TL;DR

This paper introduces COMPARE, a new logical operator for relational databases that simplifies and accelerates complex subset comparison queries, significantly improving performance over existing methods.

Contribution

The paper proposes COMPARE, a novel logical operator for relational databases, and extends SQL Server with optimization techniques to enhance comparison query performance.

Findings

COMPARE achieves significant speedup over existing approaches

Optimizations exploit COMPARE semantics for better performance

Implementation in SQL Server demonstrates practical benefits

Abstract

Data analysis often involves comparing subsets of data across many dimensions for finding unusual trends and patterns. While the comparison between subsets of data can be expressed using SQL, they tend to be complex to write, and suffer from poor performance over large and high-dimensional datasets. In this paper, we propose a new logical operator COMPARE for relational databases that concisely captures the enumeration and comparison between subsets of data and greatly simplifies the expressing of a large class of comparative queries. We extend the database engine with optimization techniques that exploit the semantics of COMPARE to significantly improve the performance of such queries. We have implemented these extensions inside Microsoft SQL Server, a commercial DBMS engine. Our extensive evaluation on synthetic and real-world datasets shows that COMPARE results in a significant speedup over existing approaches, including physical plans generated by today's database systems, user-defined function (UDF), as well as middleware solutions that compare subsets outside the databases.