Database management system performance comparisons: A systematic literature review

TL;DR

This paper systematically reviews existing studies comparing database management systems (DBMS) performance, highlighting issues with testing methods and reporting practices that hinder real-world applicability and reproducibility.

Contribution

It synthesizes prior research on DBMS performance comparisons and offers recommendations to improve testing practices and reporting standards.

Findings

Performance tests often do not reflect real-world scenarios

Reporting of test details is frequently insufficient

There is a need for standardized testing approaches

Abstract

Efficiency has been a pivotal aspect of the software industry since its inception, as a system that serves the end-user fast, and the service provider cost-efficiently benefits all parties. A database management system (DBMS) is an integral part of effectively all software systems, and therefore it is logical that different studies have compared the performance of different DBMSs in hopes of finding the most efficient one. This study systematically synthesizes the results and approaches of studies that compare DBMS performance and provides recommendations for industry and research. The results show that performance is usually tested in a way that does not reflect real-world use cases, and that tests are typically reported in insufficient detail for replication or for drawing conclusions from the stated results.