To get good student ratings should you only teach programming courses? Investigation and implications of student evaluations of teaching in a software engineering context

TL;DR

This study investigates how the topic of a software engineering course influences student evaluation of teaching scores, revealing biases where programming courses tend to receive higher ratings than others.

Contribution

It provides empirical evidence that course topics significantly affect student evaluation scores in software engineering education, highlighting potential biases.

Findings

Programming courses received higher SET ratings.

Course topic influences evaluation scores.

Biases in student evaluations are confirmed.

Abstract

Student evaluations of teaching (SET) are commonly used in universities for assessing teaching quality. However, previous literature shows that in software engineering students tend to rate certain topics higher than others: In particular students tend to value programming and software construction over software design, software engineering models and methods, or soft skills. We hypothesize that these biases also play a role in SET responses collected from students. The objective of this study is to investigate how the topic of a software engineering course affects the SET metrics. We accomplish this by performing multilevel regression analysis on SET data collected in a software engineering programme. We analyzed a total of 1295 student evaluations from 46 university courses in a Finnish university. The results of the analysis verifies that the student course evaluations exhibit similar biases as distinguished by previous software engineering education research. The type of the course can predict a higher SET rating. In our dataset, software construction and programming courses received higher SET ratings compared to courses on software engineering processes, models, and methods.