Beyond Surveys: Analyzing Software Development Artifacts to Assess Teaching Efforts

TL;DR

This paper proposes analyzing software development artifacts, such as repositories, to assess student behavior and curriculum effectiveness in teaching agile methodologies, providing insights beyond traditional surveys.

Contribution

It introduces a novel approach of using software repository analysis to evaluate student workflows and curriculum impact in software engineering education.

Findings

Surveys showed positive attitudes but couldn't measure workflow adaptation.

Repository analysis revealed differences in workflows between Scrum and Kanban.

The approach provides detailed insights into student learning behaviors.

Abstract

This Innovative Practice Full Paper presents an approach of using software development artifacts to gauge student behavior and the effectiveness of changes to curriculum design. There is an ongoing need to adapt university courses to changing requirements and shifts in industry. As an educator it is therefore vital to have access to methods, with which to ascertain the effects of curriculum design changes. In this paper, we present our approach of analyzing software repositories in order to gauge student behavior during project work. We evaluate this approach in a case study of a university undergraduate software development course teaching agile development methodologies. Surveys revealed positive attitudes towards the course and the change of employed development methodology from Scrum to Kanban. However, surveys were not usable to ascertain the degree to which students had adapted their workflows and whether they had done so in accordance with course goals. Therefore, we analyzed students' software repository data, which represents information that can be collected by educators to reveal insights into learning successes and detailed student behavior. We analyze the software repositories created during the last five courses, and evaluate differences in workflows between Kanban and Scrum usage.