Cybersecurity Knowledge and Skills Taught in Capture the Flag Challenges

TL;DR

This paper analyzes 15,963 Capture the Flag challenge solutions to assess their alignment with formal cybersecurity curricula, revealing a focus on technical skills and a need to incorporate human aspects for comprehensive training.

Contribution

It provides a mapping of challenge solutions to established cybersecurity curricula and highlights gaps in addressing non-technical skills.

Findings

Cryptography and network security are prominent topics.

Human aspects like social engineering are underrepresented.

Challenges should include non-technical skills to better prepare for cyber threats.

Abstract

Capture the Flag challenges are a popular form of cybersecurity education, where students solve hands-on tasks in an informal, game-like setting. The tasks feature diverse assignments, such as exploiting websites, cracking passwords, and breaching unsecured networks. However, it is unclear how the skills practiced by these challenges match formal cybersecurity curricula defined by security experts. We explain the significance of Capture the Flag challenges in cybersecurity training and analyze their 15,963 textual solutions collected since 2012. Based on keywords in the solutions, we map them to well-established ACM/IEEE curricular guidelines to understand which skills the challenges teach. We study the distribution of cybersecurity topics, their variance in different challenge formats, and their development over the past years. The analysis showed the prominence of technical knowledge about cryptography and network security, but human aspects, such as social engineering and cybersecurity awareness, are neglected. We discuss the implications of these results and relate them to contemporary literature. Our results indicate that future Capture the Flag challenges should include non-technical aspects to address the current advanced cyber threats and attract a broader audience to cybersecurity.