Adding semantics to measurements: Ontology-guided, systematic performance analysis

TL;DR

This paper introduces an ontology-guided framework for systematic performance analysis of complex software systems, demonstrated through a blockchain platform using standard benchmarking.

Contribution

It presents a novel ontology-based method to enhance empirical performance evaluation, addressing the lack of model-based support in complex system analysis.

Findings

Ontology-guided evaluation improves analysis rigor

Applied to Hyperledger Fabric with TPC-C benchmark

Demonstrates systematic performance assessment

Abstract

The design and operation of modern software systems exhibit a shift towards virtualization, containerization and service-based orchestration. Performance capacity engineering and resource utilization tuning become priority requirements in such environments. Measurement-based performance evaluation is the cornerstone of capacity engineering and designing for performance. Moreover, the increasing complexity of systems necessitates rigorous performance analysis approaches. However, empirical performance analysis lacks sophisticated model-based support similar to the functional design of the system. The paper proposes an ontology-based approach for facilitating and guiding the empirical evaluation throughout its various steps. Hyperledger Fabric (HLF), an open-source blockchain platform by the Linux Foundation, is modelled and evaluated as a pilot example of the approach, using the standard TPC-C performance benchmark workload.