Similarity Group-by Operators for Multi-dimensional Relational Data

TL;DR

This paper introduces two new similarity-based group-by operators for multidimensional data in SQL, enabling more meaningful grouping by considering attribute correlations, with minimal overhead and significant performance improvements.

Contribution

The paper presents novel multidimensional similarity group-by operators, addressing limitations of existing methods by considering attribute correlations and demonstrating efficient implementation in PostgreSQL.

Findings

Achieves up to 1000x performance improvement over baseline methods.

Minimal overhead with execution times comparable to standard SQL Group-by.

Effective in real-world datasets like TPC-H and social check-in data.

Abstract

The SQL group-by operator plays an important role in summarizing and aggregating large datasets in a data analytic stack.While the standard group-by operator, which is based on equality, is useful in several applications, allowing similarity aware grouping provides a more realistic view on real-world data that could lead to better insights. The Similarity SQL-based Group-By operator (SGB, for short) extends the semantics of the standard SQL Group-by by grouping data with similar but not necessarily equal values. While existing similarity-based grouping operators efficiently materialize this approximate semantics, they primarily focus on one-dimensional attributes and treat multidimensional attributes independently. However, correlated attributes, such as in spatial data, are processed independently, and hence, groups in the multidimensional space are not detected properly. To address this problem, we introduce two new SGB operators for multidimensional data. The first operator is the clique (or distance-to-all) SGB, where all the tuples in a group are within some distance from each other. The second operator is the distance-to-any SGB, where a tuple belongs to a group if the tuple is within some distance from any other tuple in the group. We implement and test the new SGB operators and their algorithms inside PostgreSQL. The overhead introduced by these operators proves to be minimal and the execution times are comparable to those of the standard Group-by. The experimental study, based on TPC-H and a social check-in data, demonstrates that the proposed algorithms can achieve up to three orders of magnitude enhancement in performance over baseline methods developed to solve the same problem.