Towards a Unified Architecture for in-RDBMS Analytics

TL;DR

This paper proposes a unified architecture for in-database analytics that simplifies integration, improves performance, and enables generic optimization across multiple techniques in RDBMSs.

Contribution

It introduces a unified architecture for in-database analytics, enabling generic performance optimization and easy integration of new techniques with minimal code changes.

Findings

Achieves competitive or higher performance compared to native tools

Requires only a few dozen lines of code to add new techniques

Demonstrates feasibility across commercial and open-source RDBMSs

Abstract

The increasing use of statistical data analysis in enterprise applications has created an arms race among database vendors to offer ever more sophisticated in-database analytics. One challenge in this race is that each new statistical technique must be implemented from scratch in the RDBMS, which leads to a lengthy and complex development process. We argue that the root cause for this overhead is the lack of a unified architecture for in-database analytics. Our main contribution in this work is to take a step towards such a unified architecture. A key benefit of our unified architecture is that performance optimizations for analytics techniques can be studied generically instead of an ad hoc, per-technique fashion. In particular, our technical contributions are theoretical and empirical studies of two key factors that we found impact performance: the order data is stored, and parallelization of computations on a single-node multicore RDBMS. We demonstrate the feasibility of our architecture by integrating several popular analytics techniques into two commercial and one open-source RDBMS. Our architecture requires changes to only a few dozen lines of code to integrate a new statistical technique. We then compare our approach with the native analytics tools offered by the commercial RDBMSes on various analytics tasks, and validate that our approach achieves competitive or higher performance, while still achieving the same quality.