Evaluating the Dynamic Behavior of Database Applications

TL;DR

This paper introduces the Dynamic Evaluation Framework (DEF) and its instantiation DoEF for simulating changing access patterns in object databases, enabling assessment of dynamic clustering algorithms and object store performance.

Contribution

It presents a flexible, extensible framework for evaluating database performance under dynamic access patterns, addressing limitations of static benchmarks.

Findings

Flexible conservative re-clustering improves adaptability to access pattern changes.

DoEF effectively evaluates clustering algorithms' responsiveness to dynamic access patterns.

DoEF reveals performance issues in real-world object stores like Platypus.

Abstract

This paper explores the effect that changing access patterns has on the performance of database management systems. Changes in access patterns play an important role in determining the efficiency of key performance optimization techniques, such as dynamic clustering, prefetching, and buffer replacement. However, all existing benchmarks or evaluation frameworks produce static access patterns in which objects are always accessed in the same order repeatedly. Hence, we have proposed the Dynamic Evaluation Framework (DEF) that simulates access pattern changes using configurable styles of change. DEF has been designed to be open and fully extensible (e.g., new access pattern change models can be added easily). In this paper, we instantiate DEF into the Dynamic object Evaluation Framework (DoEF) which is designed for object databases, i.e., object-oriented or object-relational databases such as multi-media databases or most XML databases.The capabilities of DoEF have been evaluated by simulating the execution of four different dynamic clustering algorithms. The results confirm our analysis that flexible conservative re-clustering is the key in determining a clustering algorithm's ability to adapt to changes in access pattern. These results show the effectiveness of DoEF at determining the adaptability of each dynamic clustering algorithm to changes in access pattern in a simulation environment. In a second set of experiments, we have used DoEF to compare the performance of two real-life object stores : Platypus and SHORE. DoEF has helped to reveal the poor swapping performance of Platypus.