Write a Line: Tests with Answer Templates and String Completion Hints for Self-Learning in a CS1 Course

TL;DR

This study explores using regular-expression-based questions with string completion hints in a CS1 course, demonstrating high accuracy in grading and reducing instructor workload over four years with 497 students.

Contribution

It introduces a novel approach of using regex templates and hints for self-learning in programming education, showing its effectiveness and impact on teaching workload.

Findings

Regex questions achieved over 99% precision and recall.

Formative quizzes reduced instructor assistance significantly.

Some students used quizzes mainly to learn answers.

Abstract

One of the important scaffolding tasks in programming learning is writing a line of code performing the necessary action. This allows students to practice skills in a playground with instant feedback before writing more complex programs and increases their proficiency when solving programming problems. However, answers in the form of program code have high variability. Among the possible approaches to grading and providing feedback, we chose template matching. This paper reports the results of using regular-expression-based questions with string completion hints in a CS1 course for 4 years with 497 students. The evaluation results show that Perl-compatible regular expressions provide good precision and recall (more than 99\%) when used for questions requiring writing a single line of code while being able to provide string-completion feedback regardless of how wrong the initial student's answer is. After introducing formative quizzes with string-completion hints to the course, the number of questions that teachers and teaching assistants received about questions in the formative quizzes dropped considerably: most of the training question attempts resulted in finding the correct answer without help from the teaching staff. However, some of the students use formative quizzes just to learn correct answers without actually trying to answer the questions.