Using Sentiment Analysis to Investigate Peer Feedback by Native and Non-Native English Speakers

TL;DR

This study investigates how native and non-native English speakers differ in peer feedback sentiment and ratings in online U.S. computer science courses, revealing nuanced effects of language background on feedback dynamics.

Contribution

It provides a novel analysis of peer feedback sentiment differences based on language background using advanced sentiment analysis models in an educational context.

Findings

Native speakers rate feedback less favorably.

Non-native speakers write more positive feedback but receive less positive sentiment.

Language background influences peer feedback experiences in complex ways.

Abstract

Graduate-level CS programs in the U.S. increasingly enroll international students, with 60.2 percent of master's degrees in 2023 awarded to non-U.S. students. Many of these students take online courses, where peer feedback is used to engage students and improve pedagogy in a scalable manner. Since these courses are conducted in English, many students study in a language other than their first. This paper examines how native versus non-native English speaker status affects three metrics of peer feedback experience in online U.S.-based computing courses. Using the Twitter-roBERTa-based model, we analyze the sentiment of peer reviews written by and to a random sample of 500 students. We then relate sentiment scores and peer feedback ratings to students' language background. Results show that native English speakers rate feedback less favorably, while non-native speakers write more positively but receive less positive sentiment in return. When controlling for sex and age, significant interactions emerge, suggesting that language background plays a modest but complex role in shaping peer feedback experiences.