The Effect of Security Education and Expertise on Security Assessments: the Case of Software Vulnerabilities

TL;DR

This study investigates how security education and experience influence the accuracy of software vulnerability assessments, revealing that individual skills and knowledge composition are more impactful than formal experience.

Contribution

It provides empirical insights into the relationship between assessor background and assessment quality, emphasizing the importance of skill combination over experience.

Findings

Individual characteristics influence assessment accuracy more than experience.

Skills and knowledge composition significantly affect assessment quality.

Professional expertise's advantage depends on skills and available information.

Abstract

In spite of the growing importance of software security and the industry demand for more cyber security expertise in the workforce, the effect of security education and experience on the ability to assess complex software security problems has only been recently investigated. As proxy for the full range of software security skills, we considered the problem of assessing the severity of software vulnerabilities by means of a structured analysis methodology widely used in industry (i.e. the Common Vulnerability Scoring System (\CVSS) v3), and designed a study to compare how accurately individuals with background in information technology but different professional experience and education in cyber security are able to assess the severity of software vulnerabilities. Our results provide some structural insights into the complex relationship between education or experience of assessors and the quality of their assessments. In particular we find that individual characteristics matter more than professional experience or formal education; apparently it is the \emph{combination} of skills that one owns (including the actual knowledge of the system under study), rather than the specialization or the years of experience, to influence more the assessment quality. Similarly, we find that the overall advantage given by professional expertise significantly depends on the composition of the individual security skills as well as on the available information.