BIPeC: A Combined Change-Point Analyzer to Identify Performance Regressions in Large-scale Database Systems

TL;DR

This paper introduces BIPeC, an automated method combining Bayesian inference and PELT algorithm to efficiently detect performance regressions in large-scale database systems like SAP HANA, reducing manual effort and errors.

Contribution

It presents a novel automated approach that integrates Bayesian inference with PELT for precise and efficient change-point detection in performance metrics.

Findings

High detection accuracy with minimized false negatives

Faster performance regression identification compared to previous methods

Enhances reliability and sustainability of large-scale database systems

Abstract

Performance testing in large-scale database systems like SAP HANA is a crucial yet labor-intensive task, involving extensive manual analysis of thousands of measurements, such as CPU time and elapsed time. Manual maintenance of these metrics is time-consuming and susceptible to human error, making early detection of performance regressions challenging. We address these issues by proposing an automated approach to detect performance regressions in such measurements. Our approach integrates Bayesian inference with the Pruned Exact Linear Time (PELT) algorithm, enhancing the detection of change points and performance regressions with high precision and efficiency compared to previous approaches. Our method minimizes false negatives and ensures SAP HANA's system's reliability and performance quality. The proposed solution can accelerate testing and contribute to more sustainable performance management practices in large-scale data management environments.