Dynamic Physiological Partitioning on a Shared-nothing Database Cluster

TL;DR

This paper introduces a dynamic physiological partitioning scheme for shared-nothing database clusters that adapts data distribution based on workload and energy constraints, enabling energy-efficient operation without disrupting ongoing transactions.

Contribution

It adapts physiological partitioning for database clusters, allowing easy reorganization of data to optimize energy use while maintaining transaction availability.

Findings

Significant energy savings achieved without major performance loss.

Dynamic repartitioning reduces data migration overhead.

Approach outperforms traditional physical and logical partitioning methods.

Abstract

Traditional DBMS servers are usually over-provisioned for most of their daily workloads and, because they do not show good-enough energy proportionality, waste a lot of energy while underutilized. A cluster of small (wimpy) servers, where its size can be dynamically adjusted to the current workload, offers better energy characteristics for these workloads. Yet, data migration, necessary to balance utilization among the nodes, is a non-trivial and time-consuming task that may consume the energy saved. For this reason, a sophisticated and easy to adjust partitioning scheme fostering dynamic reorganization is needed. In this paper, we adapt a technique originally created for SMP systems, called physiological partitioning, to distribute data among nodes, that allows to easily repartition data without interrupting transactions. We dynamically partition DB tables based on the nodes' utilization and given energy constraints and compare our approach with physical partitioning and logical partitioning methods. To quantify possible energy saving and its conceivable drawback on query runtimes, we evaluate our implementation on an experimental cluster and compare the results w.r.t. performance and energy consumption. Depending on the workload, we can substantially save energy without sacrificing too much performance.