Object Database Benchmarks

TL;DR

Object database benchmarks are essential tools for evaluating, comparing, and tuning OODBMS performance, aiding both designers and users in system development and selection.

Contribution

This paper reviews the importance and components of benchmarks for object-oriented database systems, emphasizing their role in performance evaluation and system optimization.

Findings

Benchmarks help compare OODBMS performance.

They assist in system tuning and hardware configuration.

Performance evaluation guides system design and selection.

Abstract

The need for performance measurement tools appeared soon after the emergence of the first Object-Oriented Database Management Systems (OODBMSs), and proved important for both designers and users (Atkinson \& Maier, 1990). Performance evaluation is useful to designers to determine elements of architecture and more generally to validate or refute hypotheses regarding the actual behavior of an OODBMS. Thus, performance evaluation is an essential component in the development process of well-designed and efficient systems. Users may also employ performance evaluation, either to compare the efficiency of different technologies before selecting an OODBMS or to tune a system.Performance evaluation by experimentation on a real system is generally referred to as benchmarking. It consists in performing a series of tests on a given OODBMS to estimate its performance in a given setting. Benchmarks are generally used to compare the global performance of OODBMSs, but they can also be exploited to illustrate the advantages of one system or another in a given situation, or to determine an optimal hardware configuration. Typically, a benchmark is constituted of two main elements: a workload model constituted of a database and a set of read and write operations to apply on this database, and a set of performance metrics.