Bayesian Hierarchical Modelling for Tailoring Metric Thresholds

TL;DR

This paper introduces a Bayesian hierarchical modeling approach to improve local prediction accuracy in software metrics thresholds by leveraging global data, demonstrating significant error reduction over global models.

Contribution

It applies Bayesian hierarchical modeling to software metrics, enabling better local predictions while maintaining global context, which is simpler than clustering or transfer learning.

Findings

Hierarchical model reduces prediction error by up to 50%.

Supports cross-project comparisons with preserved local context.

Demonstrates effectiveness through a replication study.

Abstract

Software is highly contextual. While there are cross-cutting `global' lessons, individual software projects exhibit many `local' properties. This data heterogeneity makes drawing local conclusions from global data dangerous. A key research challenge is to construct locally accurate prediction models that are informed by global characteristics and data volumes. Previous work has tackled this problem using clustering and transfer learning approaches, which identify locally similar characteristics. This paper applies a simpler approach known as Bayesian hierarchical modeling. We show that hierarchical modeling supports cross-project comparisons, while preserving local context. To demonstrate the approach, we conduct a conceptual replication of an existing study on setting software metrics thresholds. Our emerging results show our hierarchical model reduces model prediction error compared to a global approach by up to 50%.