Analyzing and Calibrating Risk Assessment by Software Developers

TL;DR

This paper investigates how software developers perceive risks, identifies subjective biases, and proposes a mathematical calibration model to improve the accuracy of risk assessments in software project management.

Contribution

It clarifies the factors influencing developers' risk perception and introduces a calibration method to enhance risk assessment reliability.

Findings

Risk perception is influenced by 'unknown' and 'dread' factors.

Developer experience can affect risk evaluation.

The calibration model reduces average absolute error to 0.20.

Abstract

In software project management, risk management is a critical factor. Project managers use existing lists of risk or perform brainstorming to identify the risks. However, it is not easy to perceive all the risks objectively. As a result, some risks are perceived based on subjective impression, which leads to risk biases. So, our goals are (i) We clarify the risk perception of developers to enhance the reliability of the brainstorming, and (ii) we calibrate the risk assessment based on a mathematical model to make more accurate risk list. In the analysis, we collected data concerning the risk perception of 69 professional software developers via a questionnaire. The average number of years of experience among these professionals was 18.3. Using the dataset, we applied factor analysis to clarify the factors that affect the evaluation of risk impact. The questionnaire was based on the risk perception theory established by Slovic, in which "dread" and "unknown" are the major factor of risk perception. The analysis result shows that (i) risk experience (i.e., whether a developer actually faced the risk or not) sometimes affects risk assessment (evaluation of risk impact), (ii) risk perception is considered to be based on unknown and dread factors, and (iii) risk assessment can be calibrated by a mathematical model (the average absolute error was 0.20).