CryptoScratch: Developing and evaluating a block-based programming tool for teaching K-12 cryptography education using Scratch

TL;DR

CryptoScratch is a visual programming tool built on Scratch that helps K-12 students learn cryptography by implementing algorithms and schemes, with an interactive task block for guided learning and feedback.

Contribution

This paper introduces CryptoScratch, a novel block-based cryptography teaching framework integrated into Scratch, including a task block for instruction and assessment.

Findings

Approximately 60% of students quickly grasped complex cryptography concepts.

Around 90% of students felt comfortable with cryptography concepts and use-cases.

Initial usability study with middle-school students showed positive engagement and learning outcomes.

Abstract

This paper presents the design, implementation, and evaluation of a new framework called CryptoScratch, which extends the Scratch programming environment with modern cryptographic algorithms (e.g., AES, RSA, SHA-256) implemented as visual blocks. Using the simple interface of CryptoScratch, K-12 students can study how to use cryptographic algorithms for services like confidentiality, authentication, and integrity protection; and then use these blocks to build complex modern cryptographic schemes (e.g., Pretty Good Privacy, Digital Signatures). In addition, we present the design and implementation of a Task Block that provides students instruction on various cryptography problems and verifies that they have successfully completed the problem. The task block also generates feedback, nudging learners to implement more secure solutions for cryptographic problems. An initial usability study was performed with 16 middle-school students where students were taught basic cryptographic concepts and then asked to complete tasks using those concepts. Once students had knowledge of a variety of basic cryptographic algorithms, they were asked to use those algorithms to implement complex cryptographic schemes such as Pretty Good Privacy and Digital Signatures. Using the successful implementation of the cryptographic and task blocks in Scratch, the initial testing indicated that $\approx 60\%$ of the students could quickly grasp and implement complex cryptography concepts using CryptoScratch, while $\approx 90\%$ showed comfort with cryptography concepts and use-cases. Based on the positive results from the initial testing, a larger study of students is being developed to investigate the effectiveness across the socioeconomic spectrum.

Tables

Table 1. TABLE I: Students’ Grade and Gender Distribution

Grade	N	%
6	3	18.75
7	4	25.00
8	5	31.25
9	3	18.75
10	1	6.25

Table 2. TABLE II: Scratch Experience

Scratch Usage	N	%
Never	6	37.50
A little	5	31.25
Some	4	25.00
A lot	1	6.25

Table 3. TABLE III: Experience with CryptoScratch Challenges

Student Response	Q1	Q2	Q3
Completely disagree	0	0	0
Somewhat disagree	2	1	0
Neutral	5	2	2
Somewhat agree	4	7	7
Completely agree	5	6	7

Full text

I Introduction

In today’s data-driven world, cybersecurity is crucial for data protection. Many government and private organizations are vulnerable to threats from external actors, and the quality and quantity of cybercrimes keep getting more sophisticated every year. One example of a sophisticated attack is a ransomware attack that surged to record levels in 2021 [1]. A disturbing trend of cyberwars is also enfolding between nations resulting in many new disruptive and devastating attack strategies. To defend against such sophisticated attacks, we need a highly qualified cybersecurity workforce. Unfortunately, according to surveys and interviews conducted worldwide, there is a vast and growing cybersecurity talent shortage. For instance, as per a cybersecurity workforce study by (ISC)2, the US has a shortage of 359,236 cybersecurity professionals and such gaps are common across the world with a global requirement of about 3.12 million professionals [2]. According to ISACA, 61% of companies feel that they are somewhat or significantly understaffed in cybersecurity and 55% currently have unfilled cybersecurity positions [3].

In order to address this deficit, government, academia and industry professionals have all stepped up and introduced many exciting and encouraging programs to bring awareness and develop an interest in cybersecurity among children from diverse backgrounds (e.g., GenCyber [4], CyberPatriots [5], NICE [6], PLTW [7]). We believe that these programs are and will keep encouraging many students to take up cybersecurity as their profession. Studies in the past also back our belief by showing evidence that increasing awareness of cybersecurity among K-12 students has raised their interest in learning more about cybersecurity career paths [8]. In addition to these initiatives, Computer Science curricular developments across schools and universities have been adding cybersecurity as a necessary component [9, 10, 11] to their curriculum in addition to creating camps (e.g., GenCyber) to teach cybersecurity to K-12 students [4].

Even after updates to the K-12 curriculum and the introduction of camps, only 5% of high-school students and 1% of middle-school students in the US are aware of topics such as Cryptography [8]. Cybersecurity education is still highly underrepresented in middle-school curriculum [12]. As Cryptography is an integral part of cybersecurity, systems and projects such as CryptoClub [13], programs run by the National Cryptologic Museum [14] and the K-12 Classroom Support from NSA [15] have created resources to effectively expose students to cryptography and encourage them to explore cryptographic algorithms. Unfortunately, due to the absence of tools for hands-on practice, these programs are often limited to simple ciphers such as the Caesar cipher. We believe that the intuition behind teaching the Caesar cipher is its simplicity and its ability to be computed by hand or using simple tools. Teaching and practicing modern cryptographic algorithms like AES and RSA, on the other hand, requires the knowledge of complex tools or programming languages such as Python that may not be intuitive for young children. In addition, these algorithms have mathematical premises (e.g., finite fields, modular arithmetic) and are often taught to students using these premises, which is not feasible in the K-12 curriculum.

This paper focuses on two related research questions: (RQ1) can we create a simple, intuitive, and visual block-based tool for K-12 students to learn and practice modern cryptographic algorithms without delving into the mathematical complexities of these algorithms? and (RQ2) how well can K-12 students understand and implement complex cryptographic schemes that drive modern systems and applications (e.g., PKI, Digital Signatures)? We note that RQ1 focuses on implementing a practical hands-on tool compatible with the systems mentioned earlier, and RQ2 delves into understanding how the usage of this tool can impact the computation thinking of students and the learning of cryptographic algorithms and schemes.

To answer RQ1, we have implemented a novel framework called CryptoScratch as an extension of the Scratch [16] programming language. CryptoScratch implements modern cryptographic algorithms such as AES, RSA, and SHA-256 as visual blocks. These blocks abstract away the mathematical details of the algorithms to enable students to interact with these blocks easily and learn cryptography without learning the mathematical premises behind the algorithms. Figure 1 shows an example of the implementation of symmetric cryptography visually using the AES Blocks in CryptoScratch. The CryptoScratch framework also implements a Task Block that performs two functions to nudge students towards writing more secure cryptographic schemes. First, it enables educators to build new problems/challenges that students can solve to improve their cryptography knowledge. Second, it enables students to get feedback about their solution’s security and accuracy, enabling them to try different cryptographic schemes and analyze their outputs. Figure 2 shows an example of a task block that checks a student’s solution for accuracy of the Pretty Good Privacy (PGP) scheme. We note that the CryptoScratch framework has been implemented as a Scratch extension and will be made open-source and publicly available through the MIT Scratch website. Given its simplicity and wide availability, we also emphasize that the framework can be easily integrated into the K-12 curriculum and the aforementioned programs (e.g., CryptoClub, GenCyber) to enable students to practice the cryptographic concepts taught in their classes and in these programs.

To answer RQ2, we performed a preliminary evaluation of CryptoScratch through a 3-day workshop comprising 16 middle-school students. The goal was to understand how well students can understand cryptography, understand different security services, and build complex cryptographic schemes using CryptoScratch. The workshop taught these students several concepts such as symmetric and asymmetric cryptography, hashing algorithms, services such as confidentiality, integrity protection, and authentication, and modern schemes such as PGP and Digital Signatures. We also gave the students eight challenges and analyzed their solutions. Based on a pre-post survey conducted with the 16 students and their challenge solutions, we observed that $\approx~{}60\%$ of the students could easily understand and implement the above complex schemes using CryptoScratch ($\approx~{}90~{}-~{}100\%$ for simpler challenges). Meanwhile, $\approx 90\%$ showed increased comfort with cryptography after the end of the workshop. These numbers indicate that CryptoScratch can be employed as an effective tool to teach cryptography in K-12 classrooms worldwide.

In summary, we make the following contributions:

• •

We have designed and implemented a new framework called CryptoScratch as a Scratch extension that implements a selection of modern cryptographic algorithms such as AES, RSA, and SHA-256 as visual blocks on the Scratch platform.

• •

We have designed and implemented a Task Block on CryptoScratch that analyzes students’ solutions for security problems and accuracy, and nudges them towards more secure solutions.

• •

We have implemented eight cryptographic learning challenges using the new Task Block that students can use to learn different aspects of cryptography such as symmetric cryptography, asymmetric cryptography, hashing algorithms, and combining cryptosystems.

• •

We evaluated the usability and effectiveness of CryptoScratch through a workshop focused on teaching cryptography using CryptoScratch to 16 middle-school students. We also conducted pre- and post-test user surveys. Approximately $60\%$ of students showed comfort implementing complex cryptographic schemes, while $\approx 90\%$ showed comfort with cryptography concepts.

The rest of this paper is structured as follows. In Section II, we discuss some of the related works that focus on cybersecurity and cryptography education with K-12 students and compare our CryptoScratch framework with these prior works. Section III focuses on the design and implementation of the CryptoScratch framework, the design of the Task Block, and the description of cryptographic challenges implemented in CryptoScratch. Section IV describes the design, the metrics, and the results of the user study that was conducted with 16 middle-school students to evaluate the effectiveness of CryptoScratch. We conclude and discuss future work in Section V.

II Related Work

We acknowledge that many initiatives exist that focus on cybersecurity education and awareness for diverse socioeconomic groups, education of women and other underrepresented groups [17, 18, 19, 20]. This section highlights only a subset of these initiatives. Our focus is on demonstrating how these initiatives encourage cybersecurity education and how CryptoScratch can enable the hands-on practice of cryptographic algorithms and schemes during these initiatives. GenCyber [4] is a camp program funded by the NSF and NSA where universities host K-12 students and attempt to bring awareness of and excitement for cybersecurity. We note that our university also conducts GenCyber summer camps where, in previous camps, cryptographic algorithms and schemes were taught using the Python programming language. We will be switching to CryptoScratch in our future camps to simplify the cryptography concepts for students as Scratch is a platform that many K-12 students express as the most intuitive.

Other initiatives such as Hacker High school from the Institute for Security and Open Methodologies (ISECOM) [21] provide a self-directed learning curriculum focused on attack and defense skills. The AFA CyberPatriot program created by the Air Force Association [22] is a program that works to inspire students to pursue careers in cybersecurity or related STEM fields. Other examples include [23] which is a week-long summer camp program for high school students addressing a broad range of cybersecurity techniques. The private sector has also made efforts to improve cybersecurity education and awareness. For example, the Bits N Bytes organization [24] is dedicated to bringing awareness of the field of cybersecurity to young students. Cybrary is a platform that provides self-paced training in cybersecurity for a wide range of skill levels [25]. Another example is [26] which presents an online learning tool for attack and defense. Works such as Cybersecurity Lab as a Service (CLaaS) [27] have also looked at creating services that can run other cybersecurity tools on their platform. We note that even though these initiatives focus on diverse cybersecurity domains, they include cryptography as one of the core components. Our analysis of these initiatives reveals that most focus on simple tools for demonstrating simple cryptographic algorithms such as Caesar cipher. In the case of complex algorithms, the education is primarily verbal and does not provide tools for hands-on practice of cryptography. The CryptoScratch framework can fill the gap for these initiatives.

Some initiatives also focus on cryptography such as the Cryptographic Protocols sections of Classical CS Unplugged [28]. Similarly, NCC Group hosts the Cryptopals Crypto Challenges [29] that provide participants the opportunity to write code to solve cryptographic problems. Cryptopals focuses on already strong programmers who want to explore cryptographic concepts independently. The CryptoClub Project [30] provides curriculum for teachers and an interactive website [13] that allow the exploration of classical cryptographic techniques. LLCipher from the Lincoln Laboratory [31] is a workshop designed to teach the theory of Cryptography to students interested in the mathematics of cybersecurity. They focus on teaching the use of classical cryptography and the theory of cryptography. Prior works on teaching cryptography to middle-school students have also used various techniques. For instance, [32] used drawings to help students learn and explain the basics of cryptography and the components of secure communications systems. González-Tablas et al. [33] used a CryptoGo card game to teach cryptography. Other works such as [34] attempted to develop visual techniques to teach public key infrastructure concepts. These works differ from ours as they either provide tools to teach the concepts or have specific use-cases that cannot be extended to learn other concepts. Meanwhile, CryptoScratch provides a generic, simple and visual tool for understanding and practicing several cryptographic algorithms and schemes. As such, CryptoScratch can be directly integrated into the above initiatives and curriculum. For instance, Cryptopals can use the CryptoScratch Task Block to attract K-12 students on their platform who may not already be strong programmers but are interested in solving complex cryptographic problems.

Several research papers also showcase the importance of cryptography education and tools for improving cybersecurity education and awareness. For instance, Svabensky et al. [35] published a systematic literature review of SIGCSE and ITiCSE conference papers from 2010 to 2019. They reported that only a small portion of the papers focused on cryptography among all sub-fields of cybersecurity education across all age groups. In particular, works focused on K-12 students were limited. While there is a gap between  [35] and now, cryptography in K-12 cybersecurity education is still underrepresented. There are also some excellent prior works to be noted, such as teaching cryptography using an interactive protocol called CYPHER [36], research on secure coding with misuse pattern detection techniques [37] and secure coding for K-12 female students [38]. In [39], the authors implemented a game where students used cryptography and cryptanalysis to solve problems and then used the clues to move to the next clue station. These works highlight the importance of tools for hands-on practice of cryptography that our CryptoScratch framework is intended to provide.

III CryptoScratch Design and Implementation

This section focuses on the design and implementation details of CryptoScratch. In particular, we outline our reasons for implementing CryptoScratch on the Scratch platform, the design of modern cryptographic algorithms as CryptoScratch Blocks, the design of the Task Block for nudging learners towards more secure solutions, and details of the challenges implemented to work with the aforementioned Task Block.

III-A Developing CryptoScratch on Scratch

The CryptoScratch framework is implemented as an extension to the Scratch programming language. We chose Scratch as the underlying environment for multiple reasons. First, Scratch is widely available via the Internet as an application-as-a-service or as a locally installed application for Windows, macOS, ChromeOS, and Android [40]. The environment is actively maintained by the Lifelong Kindergarten Group in the Media Laboratory at the Massachusetts Institute of Technology (MIT) through the Scratch Foundation [16]. Due to this wide availability of Scratch to anyone with even occasional access to the Internet, we believe that the framework and the implemented challenges can be immediately available to groups with diverse socioeconomic backgrounds, including underrepresented groups.

Second, due to its popularity, the Scratch programming language is used in many K-12 schools in the US and the world [41]. It is well-known for its visual block-based interface, intuitive environment, and features that make it easy for students to understand complex computer science constructs. We believe that the existing use of Scratch in the K-12 curriculum will help the adoption of CryptoScratch, as educators likely already have familiarity with Scratch making it easier for them to incorporate CryptoScratch into their teaching. Making the framework available to existing Scratch users will potentially increase its awareness and awareness of cryptographic algorithms, schemes, and protocols. As the platform is free for all, we note that it is also a financially sustainable solution to teach cybersecurity at the K-12 level in schools from underprivileged areas.

Third, Scratch currently has several extensions related to educational modules such as Artificial Intelligence, Machine Learning, and Embedded Programming. Currently, these efforts have not yet included cybersecurity even though several works [42, 43, 44, 45, 46, 47, 48] demonstrate that we can effectively teach cybersecurity concepts through gamification and visual feedback (both features of Scratch). This work intends to address this gap by implementing the first Scratch extension for cybersecurity dedicated to cryptography. In the future, we intend to further extend Scratch incorporating other vital aspects of cybersecurity, such as firewalls and intrusion detection systems. The incorporation of CryptoScratch on Scratch not only enables students to use modern cryptographic algorithms easily but also enables them to combine these algorithms to form complex cryptographic schemes like PGP (see Figure 2).

III-B Designing blocks for cryptographic algorithms

One of the primary goals of CryptoScratch is to introduce students to modern and secure cryptographic algorithms using a visual and intuitive interface and without focusing on the mathematics behind the algorithms. The intention behind this goal is that, at the middle-school level, students should not be expected to understand the inner workings of algorithms such as AES or RSA. Instead, they should be made aware of different types of cryptographic algorithms, the security services they provide, and use-cases. The design of the visual blocks allows for a simplified discussion by focusing on the application of cryptography while avoiding the mathematical premises of the algorithms.

The current implementation of CryptoScratch supports the following cryptographic algorithms - Caesar, AES, CRC32, SHA-256, and RSA. Figure 3 shows the default setup of the implemented blocks. We note that this set comprises both weaker and stronger cryptographic algorithms to enable students to understand the inherent security problems with weak algorithms and how the more secure modern algorithms provide a higher level of security. The industry-standard AES and RSA encryption algorithms demonstrate symmetric and asymmetric cryptography, respectively. We note that the RSA algorithm also implements blocks that enable students to generate an RSA public and private key pair. The Caesar cipher can explain cryptanalysis attacks on symmetric cryptosystems. In addition, CRC32 and SHA-256 can illustrate the concepts of one-way functions, error checking, and digital signatures, with CRC32 also providing the capability of showing collisions. We note that the development of CryptoScratch is an ongoing effort, and many other algorithms/schemes (e.g., DES, 3DES, HMAC, Diffie Hellman, ECC) will be added in the future versions of the framework. We emphasize that the framework is still practical as students can develop complex cryptographic schemes by combining the existing blocks in different ways. An example of such a scheme is Digital Signatures that combines the SHA-256 and RSA blocks as shown in Figure 4.

Within Scratch, we have implemented all the cryptographic blocks as JavaScript functions (ES6) with support for most modern browsers. Internally, these functions invoke well-known cryptographic implementations provided by the crypto-browserify npm package [49]. We note that the CryptoScratch extension is fully compatible with Scratch since it uses the guidelines provided by the Scratch community for creating new extensions [50], so that no changes are required to the Scratch platform and source code.

[TABLE]

Listing LABEL:lst:aes shows code snippets of the implementation of the ‘Encrypt [PLAINTEXT] with key [KEY] using AES’ block (shown in Figure 3, row 3). Note that we have removed other code here to focus specifically on the AES Encryption process. In this example, the text ‘Encrypt _ with key _ using AES’ denotes the text displayed in the block. Observe that the [PLAINTEXT] and [KEY] parameters are replaced by user input fields. Internally, these parameters render as string arguments to the JavaScript function defined by the opcode parameter. The opcode parameter, in our example, points to the doAesEncrypt JavaScript function that gets executed when the specific block gets executed. We also note that we have implemented all the CryptoScratch blocks as REPORTER blocks as all the algorithms accept some inputs and return a value. More precisely, the example block internally gets translated to doAesEncrypt([PLAINTEXT], [KEY]) which returns the ciphertext in hex format. The doAesEncrypt JavaScript function then invokes the appropriate browserify objects and functions on the arguments to perform AES encryption in ECB mode to produce the ciphertext. The given key/password is hashed to create a 128-bit key to be used with the AES algorithm. We chose to use a 128-bit key size to reduce computing power on the Scratch platform. The other blocks for AES decryption, RSA encryption and decryption, SHA-256 hashing, Caesar Cipher, and CRC32 were also implemented using the same methodology described for AES Encryption and are not discussed further for the sake of brevity.

III-C Designing a Task Block for automated code feedback

To the best of our knowledge, the Scratch platform does not define any blocks or functions to provide feedback to users on their code implementation. We believe that giving feedback is especially important in cryptography since mistakes in cryptographic implementations can often lead to critical security vulnerabilities. To address this gap, we have designed and implemented a new Task Block in CryptoScratch that serves two purposes. First, this block can be used as a generic wrapper to present various challenges that learners can solve to learn cryptography. We refer the readers to Section III-D for a list of challenges already implemented using the Task Block. Second, using the block to encapsulate the learner’s solution for the challenge enables this block to check insecure implementations of cryptographic algorithms and the student’s final solution. This task block is shown in Figure 5 where users can select a task that they’d like to solve, and then choose to get help or execute the task111A video of the task block and code checker is available at https://www.dropbox.com/s/235w7posvwh8bl1/CryptoScratch.mov.

The intuition behind the feedback capability of the Task Block can be demonstrated using the popular Pretty Good Privacy (PGP) scheme. Before discussing the scheme, we will introduce some notations to simplify our discussion. Let $K\{M\}$ denote the process of encrypting a secret message $M$ using a symmetric key $K$ shared between two users A and B. In the case of public-key cryptography, let $\{M\}_{B}$ denote that the secret message $M$ is encrypted using user B’s public key and $[M]_{A}$ denote that message $M$ is encrypted using user A’s private key. Using these notations, the PGP scheme for protecting the confidentiality of a large message $M$ is represented as $K\{M\}|\{K\}_{B}$. Here, sender A generates a random symmetric key $K$ to encrypt the message $M$. To send the key $K$ to receiver B, sender A encrypts $K$ using B’s public key. This scheme provides confidentiality given that an attacker cannot decrypt the key $K$ and the message $M$ without knowledge of B’s private key. If a student implements the correct solution, the Task Block checks the accuracy of their solution by (1) decrypting $\{K\}_{B}$ first to obtain $K$ using B’s private key, (2) decrypting $M^{’}=K\{M\}$ to obtain the message $M^{’}$, and (3) verifying that the challenge message $M$ and the decrypted message $M^{’}$ are the same. For a correct solution, the Sprite displays a success message to the student as shown in Figure 6(a). However, a common mistake for learners new to cryptography stems from confusion between using the private/public key for encryption. If a learner makes a mistake and uses the wrong key for encryption, the scheme will become $K\{M\}|[K]_{A}$. This scheme no longer provides confidentiality as an attacker can use A’s public key to decrypt the key $K$ and then use $K$ to decrypt the secret message $M$. The Task Block is designed to display a warning to the learner as shown in Figure 6(b), to make them aware of the mistake and nudge them towards a more secure solution.

The PGP example above showcases one security problem (i.e., confidentiality breach) that the Task Block can detect. We note that the design of this block enables the detection of many different forms of security problems. Let’s take another example of a Digital Signature of a message $M$ that can be represented as $M|[H(M)]_{A}$, where $H(M)$ is the hash of message $M$ using a secure hash function $H$. In this specific case, the Task Block can detect problems such as a learner using a public key instead of a private key for signing the message $M|\{H(M)\}_{B}$ (authentication flaw); as well as cases where a learner may use a collision-prone insecure function such as CRC32 (signature spoofing). We note that the above checks and others are already implemented into CryptoScratch, and we intend to keep adding more checks in future revisions.

To make the above checks feasible, the Task Block has specific sections as shown in Figure 7. The top section of the block ‘Task Id - [MODE] [RESULT]’ initializes the task by setting up the parameters required for the successful completion of a task. Currently, the [MODE] parameter supports two execution modes - ‘HELP’ to display help information to the student for solving the task and ‘EXECUTE’ that executes the student’s solution for the task. The [RESULT] parameter is a placeholder where students can place the variable that will store the final solution. The bottom section of the block ‘Result Verification’ performs the code checking for secure usage and validates the [RESULT] variable by matching the task’s solution with the learner’s solution. For instance, in the case of the PGP scheme in Figure 2, the code checker performs decryption of the student’s solution by using the PGP decryption process described before and checks whether the decrypted text matches the text specified in the task. Everything between the top and bottom sections is the learner’s solution to solve the challenge identified by the task. We note that we will also provide optional starter codes that students will download from the project’s public website.

III-D Designing cryptographic challenges on CryptoScratch

The CryptoScratch framework currently implements eight challenges designed to work with the Task Block so that students can get appropriate feedback as they solve the challenges. We plan to make the CryptoScratch source code and all the challenges publicly available on GitHub to encourage further contributions to this challenge set. All these challenges are designed to help students learn about cryptography. They present situations similar to real-world systems, albeit simplified, where cryptography offers a viable solution to secure sensitive information.

The current set of challenges range from encrypting sensitive data using AES (Task 1) and decrypting sensitive data using AES (Task 2) to demonstrate the power of symmetric cryptography, known plaintext-ciphertext cryptanalysis attack on the Caesar cipher (Task 3) to demonstrate problems with weak cryptographic algorithms, encrypting sensitive data using RSA (Task 4), and decrypting sensitive data using RSA (Task 5) to demonstrate the power of asymmetric cryptography, and hashing data using the SHA-256 algorithm (Task 6) to demonstrate the concept of message integrity. As the above challenges focus on single cryptographic algorithms, we have created two additional challenges to demonstrate to learners how to combine cryptographic algorithms to create complex cryptographic schemes. These challenges are creating and appending a digital signature to a message (Task 7) and protecting the confidentiality of a message using Pretty Good Privacy (PGP) (Task 8), both discussed in Section III-C. We note that, for the K-12 level, Tasks 7 and 8 provide sufficient challenge as (1) learners must understand the differences between symmetric cryptography, asymmetric cryptography and secure hashing algorithms, (2) learners must understand the purpose behind these cryptographic schemes, and then (3) learners must implement them in CryptoScratch by combining together several cryptographic blocks (AES, RSA, SHA-256). We have also created starter codes for each challenge that students can download from our public website for CryptoScratch and upload on the Scratch environment. Figure 8 shows an example starter code for the PGP task222The screenshots of all tasks starter code and their solutions is available at https://www.dropbox.com/sh/p7xmtpgkwaug77f/AADaLKjl9h2P8H2Pz84Qy1Baa whose solution is shown in Figure 2. As we add more blocks to CryptoScratch, we will add more challenges (e.g., Message Authentication Codes, Authenticated Encryption) to enable students to learn the basics of these algorithms and then combine them to create additional functionality using the visual interface of CryptoScratch.

IV CryptoScratch Evaluation

This section outlines the details of the user study with 16 middle-school students and the qualitative and quantitative results obtained from the study.

IV-A Design of the User Study

To better understand the impacts of CryptoScratch on K-12 students’ understanding of cryptography and its application to cybersecurity, we conducted a 3-day workshop with middle-school students333This research was approved by the authors’ university’s Institutional Review Board (IRB).. The details of the workshop, the participants and the study are given below.

Recruitment Method: We utilized a snowball sampling method starting with the immediate contacts of the research team to connect with parents of middle-school students. Using this method, 16 middle school students registered and attended the workshop. These students completed a pre-workshop survey, a post-workshop survey and submitted their solutions for the challenges as requested. We refer the readers to Section III-D for the list of challenges given to these students.

Students Distribution: Table I shows the grade and gender distribution of the students. The workshop students ranged from sixth grade to tenth grade, of which the 8th and 9th grades accounted for 50% of all the students. Among the students, 56.25% were boys and 43.75% were girls.

Training / Workshop: The recruited middle school students were given a 9-hour (3 hours * 3 days) hands-on training session (virtual) on cryptography and the CryptoScratch framework. Each day, students were given a combination of instructions on the basics of cryptography such as symmetric cryptography, asymmetric cryptography, hashing, and hands-on challenges to complete using CryptoScratch. Multiple instructors were available while students were working on the challenges to answer any general questions and solve any technical problems. We note that the first six challenges helped the students better understand how the recently introduced cryptographic concepts worked. The next two (PGP and Digital Signature) were used to assess the students’ understanding of the cryptographic concepts covered and their ability to utilize CryptoScratch to implement complex solutions requiring the combination of techniques that were taught.

IV-B Pre- and Post-Survey Evaluation of CryptoScratch

Our goal with the pre- and post-survey was to understand the workshop’s impact on the students’ inclination towards cryptography. In particular, we wanted to assess whether students enjoyed learning about cryptography and saw solving challenges using CryptoScratch as a fun activity. Using the results, we wanted to determine if it is possible to generate curiosity among K-12 students towards cybersecurity so that they consider cybersecurity as a future profession.

Pre- and Post-surveys: Before starting the workshop, we obtained documented parental consent. We then asked the students to complete a research participation assent form and pre-training survey to share their demographic information, programming comfort level, experience with Scratch, and experience with cryptography. During the workshop, we also asked them to submit solutions for specific challenges solved using CryptoScratch.

After the workshop completed, the students completed a post-training survey to provide feedback on CryptoScratch. For the post-survey, the students were asked to evaluate if CryptoScratch helped in their learning, whether they noticed improvement in their understanding of cryptography, how they found the challenges in terms of clarity and difficulty, and whether the challenges were fun to solve using CryptoScratch.

Analysis of Survey Results: The pre-survey results indicated that most of the students were new to programming and had limited experience with Scratch. For instance, Table II shows that only 5 (31.25%) students had some experience with Scratch, with only one student having significant experience with the platform. We note that six students started the workshop with no programming and Scratch experience and started learning both Scratch and CryptoScratch only during the workshop. Nearly all the 16 students had low to no experience with cryptographic algorithms and schemes.

The post-survey results indicated a positive experience of cryptography among many students. For example, 15 (93.75%) students out of the 16 reported agreement with the statement that they gained more knowledge about cryptography due to the workshop. Meanwhile, one student expressed that they were neutral about the above statement. The results also indicated a positive experience with the CryptoScratch framework and Task Block. For instance, 11 (68.75%) students reported that they found the framework easy to use and understand, and 3 (18.75%) students said they were neutral about the above statement. In contrast, 2 (12.5%) students reported not finding CryptoScratch easy to use and understand. We remind the readers that, before the workshop, 11 students mentioned that they had little to no experience with Scratch. This means that at least six of these students found the CryptoScratch framework easy to use and understand despite their limited experience with the Scratch platform.

Table III shows the results related to students’ understanding of the challenges. We note that question Q1 checks whether students found the challenges easy to complete, Q2 checks whether students found the challenges easy to understand, and Q3 checks whether students had fun solving the challenges. We observe that the responses are mainly positive, indicating that the students had a positive experience with the challenges. For instance, 9 (56.25%) students found the challenges to be easy to complete, 5 (31.25%) were neutral, and only 2 (12.5%) students found the challenges not as easy. We remind the readers that these challenges comprise two (Pretty Good Privacy and Digital Signatures) that require a deeper understanding of cryptographic algorithms and schemes and are considered hard to comprehend for students even at the university level. For Q2, a majority (13 students) reported that the challenges were easy to understand, which reveals a firmer understanding of cryptographic concepts. Meanwhile, 14 students reported that they had fun doing the challenges, while two were neutral about that statement. We note that these results indicate that students find CryptoScratch easy to understand and fun to work with and reaffirm that the tool can be effective to teach cryptography in K-12 classrooms.

We note that the sample size of this study is not sufficient to produce any statistically significant results. However, we also emphasize that these initial results encourage additional studies to evaluate the effectiveness of CryptoScratch further. A larger study of more diverse student groups (e.g., sociodemographic groups, of technology access and comfort levels, and previous programming experience) will be performed in the future with enough participants to allow a generalizable, statistically significant conclusion.

IV-C Evaluation of Student Solutions

This section performs a quantitative analysis of CryptoScratch by analyzing the solutions to the challenges that students completed during the workshop. We note that, among the eight challenges (Section III-D) completed by the students, our discussion focuses on two challenges - Pretty Good Privacy and Digital Signatures. This is because the first six challenges focused on simple usages of cryptographic algorithms such as AES, RSA, Caesar, and SHA-256 that nearly all the students could complete quickly. The PGP and Digital Signature challenges, on the other hand, were deemed harder by the students, and not all the students were able to complete them in the allocated time of 20 minutes. As such, these challenges provide us insights into whether the students gathered a solid foundation of cryptography and were able to use them to solve cryptographic problems in CryptoScratch.

In the case of the PGP and Digital Signature challenges, we note that we provided the students with starter code such as the one shown in Figure 8. Our deadline to complete each challenge was 20 minutes, an assessment by the instructors based on the completion time of the easier challenges. For Task 7 (Digital Signature), an analysis of the submissions found that 8 (66.67%) out of 12 students implemented a correct solution similar to Figure 9. Two (16.67%) students submitted an incorrect solution, and 2 (16.67%) students submitted the starter code given to them. Analyzing the two incorrect solutions revealed that the students were close to obtaining the correct results had we given them more time. For instance, one student’s error was to overwrite the original message with a hash of the message and then sign the hash of the hash. The other student submitted a partial solution where they successfully calculated the hash and signed the hash. The only additional step they needed to reach the correct solution was to concatenate the message with the signature in the result.

For Task 8 (PGP), 9 (60%) out of 15 students submitted the correct solution, 4 (26.7%) made some basic errors pointing to potential confusion between the uses of AES and RSA, and 2 (13.3%) submitted the starter code given to them. Figure 10 shows the error made by two students. In their case, they tried to encrypt the random key with the RSA public key correctly but used the AES block instead of the RSA block for encryption. We note that this error could also mean negligence from the student and may not specifically highlight confusion between symmetric and asymmetric cryptography. One solution (Figure 11), an ideal case for the Task Block, mistakenly sent the plaintext message instead of the ciphertext message. Again, this mistake could be an artifact of the time constraints on the student rather than confusion between plaintext and ciphertext. Our intuition is derived from the fact that the student did think about encrypting the message (as suggested by the EncryptedMessage variable block). Our planned larger study with more students will look at these errors more closely and understand the underlying causes of the errors. We plan to use the analysis further to improve the feedback capability of the Task Block.

V Conclusion & Future Work

This paper presented the design, implementation, and evaluation of a new framework, CryptoScratch, built as an extension of the Scratch programming language. The CryptoScratch framework implements strong cryptographic algorithms (e.g., AES, RSA, SHA-256) and weaker algorithms (e.g., Caesar cipher, CRC32) as simple, visual, and intuitive blocks. These blocks enable students to interact with cryptographic algorithms, learn weaknesses and strengths of the algorithms, and use the blocks to create more complex cryptographic schemes such as Pretty Good Privacy (PGP) and Digital Signatures. The successful development of CryptoScratch demonstrates that a visual block-based tool for teaching cryptography without the mathematical complexity is possible (RQ1). By using the CryptoScratch environment for an initial study, the ability to use CryptoScratch to help K-12 students understand and complete basic implementations of modern cryptographic systems has been demonstrated (RQ2).

We believe that the framework enables hands-on practices for cryptography and can be directly integrated into existing initiatives (e.g., GenCyber, CryptoClub) and the K-12 curriculum. We also presented the design and implementation of a new Task Block on CryptoScratch that enables students to solve cryptographic challenges to learn about cryptography. The Task Block generates feedback for student solutions to nudge them towards more secure cryptographic solutions. We note that the development of CryptoScratch is an ongoing effort, and other algorithms/schemes (e.g., HMAC, Diffie Hellman, ECC) will be added to the framework. We plan to make CryptoScratch and the Task Block open-source and publicly available through the MIT Scratch website.

We also evaluated CryptoScratch through an initial usability study with 16 middle-school students. During the study, we introduced the students to cryptography and the CryptoScratch framework. We asked them to complete a set of challenges to learn about cryptographic algorithms and then use those algorithms to implement complex cryptographic schemes. The study indicated that nearly 60% of the students could quickly grasp and implement complex cryptography concepts using CryptoScratch, while nearly 90% showed comfort with cryptography concepts and use-cases. We plan to do a larger study of more diverse student groups and enough participants to allow a generalizable, statistically significant conclusion.