Data-driven insight into the puzzle-based cybersecurity training

TL;DR

This paper introduces a visualization tool for analyzing cybersecurity puzzle training sessions, aiding educators in assessing trainee performance and identifying assignment flaws through qualitative evaluation.

Contribution

We developed the Training Analysis Tool, a novel visualization system for post-training data analysis in cybersecurity puzzle-based education.

Findings

Participants found the tool positively impacted their workflows

The tool enabled detailed trainee comparisons

Insights can inform future educational tool design

Abstract

Puzzle-based training is a common type of hands-on activity accompanying formal and informal cybersecurity education, much like programming or other IT skills. However, there is a lack of tools to help the educators with the post-training data analysis. Through a visualization design study, we designed the Training Analysis Tool that supports learning analysis of a single hands-on session. It allows an in-depth trainee comparison and enables the identification of flaws in puzzle assignments. We also performed a qualitative evaluation with cybersecurity experts and students. The participants apprised the positive influence of the tool on their workflows. Our insights and recommendations could aid the design of future tools supporting educators, even beyond cyber security.