Carrot and Stick approaches revisited when managing Technical Debt in an educational context

TL;DR

This study compares penalisation and reward strategies for managing technical debt in software engineering education, finding that reward-based approaches significantly outperform penalisation in maintaining low technical debt levels.

Contribution

It introduces and empirically evaluates reward and penalisation assessment strategies for teaching technical debt management using SonarQube metrics.

Findings

Reward strategy outperforms penalisation in 5 of 8 metrics

Statistically significant improvements with reward approach

Medium to large effect sizes observed in results

Abstract

Technical Debt management is an important aspect in the training of Software Engineering students. In this paper we study the effect of two assessment strategies in an educational context: One based on penalisation, the other based on rewards. Both are applied to assignments where the students develop a project focusing on keeping a low technical debt level, and obtaining a high quality code. We describe the design, tools and context of the strategies applied. SonarQube, a tool commonly used in production environments, is used for measuring the metrics. The penalisation strategy is based on a SonarQube quality gate. The reward strategy is based on a contest, where an automatic judge tool is devised to provide an online leaderboard with a classification based on the SonarQube metrics. An empirical study is conducted to determine which of the strategies works better to help the students/trainees keep the Technical Debt low. Statistically significant results are obtained in 5 of the 8 analysed metrics, showing that the reward strategy works much better. The effect size of the executed statistical tests is analysed, resulting in medium and large effect size in the majority of the analysed metrics.