# Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture

## TL;DR

This study shows that student evaluations of teaching are influenced more by lecturer personality traits and student biases than by teaching quality, using text and numerical data from an Australian university.

## Contribution

The paper introduces a method to link text comments with numerical ratings using sentiment topic analysis and Bayesian ordinal regression.

## Key findings

- Student SET ratings correlate more with lecturer traits like approachability than with teaching quality.
- A gender bias was found favoring the majority gender in a faculty.
- Lecturers with non-English backgrounds received lower ratings, especially from local students.

## Abstract

Student evaluations of teaching (SET) have been widely used by university staff to inform decisions on hiring and promotion. In recent years, an increasing body of research has revealed that student evaluations may be systemically affected by students’ own conscious or unconscious biases. In this article, we study a data set from an Australian university, where both numerical and text survey responses were available in large quantities. Our study directly linked comments to numerical ratings, we developed approaches to convert text to quantitative data in the form of topics and sentiment scores, and make use of Bayesian ordinal regression techniques to identify drivers of SET scores. Our analysis of text identified 6 teaching dimensions that students discuss in their comments. Our findings suggest that students’ SET ratings were correlated primarily with the personal characteristics of the lecturer (such as approachability, and being nice) than measures related to teaching dimensions such as course content and assessment. We found a positive gender effect towards the majority gender in a faculty, possibly reflecting students’ gendered expectations. Finally we found that lecturers with a non-English language background were consistently rated lower by the student population, and this effect manifests strongly in local students.