How do students test software units?

TL;DR

This study investigates how students with basic programming knowledge perceive and practice software testing, revealing misconceptions and non-systematic testing behaviors despite their beliefs of systematic testing.

Contribution

It provides empirical insights into students' testing ideas and misconceptions after an introductory programming course without formal testing education.

Findings

Students do not test systematically despite believing they do.

Most students think of test cases based solely on code.

Students attempt to create code-based test cases even in black-box scenarios.

Abstract

We gained insight into ideas and beliefs on testing of students who finished an introductory course on programming without any formal education on testing. We asked students to fill in a small survey, to do four exercises and to fill in a second survey. We interviewed eleven of these students in semi-structured interviews, to obtain more in-depth insight. The main outcome is that students do not test systematically, while most of them think they do test systematically. One of the misconceptions we found is that most students can only think of test cases based on programming code. Even if no code was provided (black-box testing), students try to come up with code to base their test cases on.