Hyper-Graph Based Database Partitioning for Transactional Workloads

TL;DR

This paper introduces a hyper-graph based system for partitioning transactional databases that minimizes distributed transactions, balances load, and incorporates user feedback for optimized performance.

Contribution

It presents a novel hyper-graph based approach for database partitioning that considers multiple constraints and allows user interaction for improved results.

Findings

Minimizes distributed transactions effectively

Balances partition sizes and workloads

Supports user feedback and visualization

Abstract

A common approach to scaling transactional databases in practice is horizontal partitioning, which increases system scalability, high availability and self-manageability. Usu- ally it is very challenging to choose or design an optimal partitioning scheme for a given workload and database. In this technical report, we propose a fine-grained hyper-graph based database partitioning system for transactional work- loads. The partitioning system takes a database, a workload, a node cluster and partitioning constraints as input and out- puts a lookup-table encoding the final database partitioning decision. The database partitioning problem is modeled as a multi-constraints hyper-graph partitioning problem. By deriving a min-cut of the hyper-graph, our system can min- imize the total number of distributed transactions in the workload, balance the sizes and workload accesses of the partitions and satisfy all the partition constraints imposed. Our system is highly interactive as it allows users to im- pose partition constraints, watch visualized partitioning ef- fects, and provide feedback based on human expertise and indirect domain knowledge for generating better partition- ing schemes.