In Search of Socio-Technical Congruence: A Large-Scale Longitudinal Study

TL;DR

This large-scale longitudinal study investigates whether socio-technical congruence, measured by developer communication and technical dependencies, impacts software quality metrics like bugs and churn, finding no significant relationship.

Contribution

The paper introduces a quantitative measure of socio-technical congruence (STMC) and applies it to 25 large open-source projects, providing a comprehensive empirical analysis.

Findings

No statistical relationship between STMC and bugs.

No statistical relationship between STMC and churn.

Socio-technical congruence does not directly influence software quality metrics.

Abstract

We report on a large-scale empirical study investigating the relevance of socio-technical congruence over key basic software quality metrics, namely, bugs and churn. In particular, we explore whether alignment or misalignment of social communication structures and technical dependencies in large software projects influences software quality. To this end, we have defined a quantitative and operational notion of socio-technical congruence, which we call socio-technical motif congruence (STMC). STMC is a measure of the degree to which developers working on the same file or on two related files, need to communicate. As socio-technical congruence is a complex and multi-faceted phenomenon, the interpretability of the results is one of our main concerns, so we have employed a careful mixed-methods statistical analysis. In particular, we provide analyses with similar techniques as employed by seminal work in the field to ensure comparability of our results with the existing body of work. The major result of our study, based on an analysis of 25 large open-source projects, is that STMC is not related to project quality measures -- software bugs and churn -- in any temporal scenario. That is, we find no statistical relationship between the alignment of developer tasks and developer communications on the one hand, and project outcomes on the other hand. We conclude that, wherefore congruence does matter as literature shows, then its measurable effect lies elsewhere.