Reflecting on the Use of the Policy-Process-Product Theory in Empirical Software Engineering

TL;DR

This paper evaluates the empirical use of the Policy-Process-Product (PPP) Theory in software engineering research, highlighting gaps and recommending better integration of the theory in future studies.

Contribution

It analyzes recent empirical studies to assess how well they evaluate the PPP Theory and offers four recommendations to improve research design and focus.

Findings

70% of studies focus on either policies/processes or products, not both

Only 33% of studies measure the relationship between policies/processes and products

Authors recommend using PPP Theory explicitly and studying feedback and feedforward relationships

Abstract

The primary theory of software engineering is that an organization's Policies and Processes influence the quality of its Products. We call this the PPP Theory. Although empirical software engineering research has grown common, it is unclear whether researchers are trying to evaluate the PPP Theory. To assess this, we analyzed half (33) of the empirical works published over the last two years in three prominent software engineering conferences. In this sample, 70% focus on policies/processes or products, not both. Only 33% provided measurements relating policy/process and products. We make four recommendations: (1) Use PPP Theory in study design; (2) Study feedback relationships; (3) Diversify the studied feedforward relationships; and (4) Disentangle policy and process. Let us remember that research results are in the context of, and with respect to, the relationship between software products, processes, and policies.