Investigating the Effects of Different Levels of User Control in an Interactive Educational Recommender System

TL;DR

This study investigates how varying levels of user control in an educational recommender system affect user perceptions and trust, demonstrating that increased control generally enhances positive perceptions.

Contribution

It provides empirical evidence on the impact of different user control levels in ERSs, highlighting the importance of profile building features for user experience.

Findings

Enabling profile building improves perceived control and trust.

Additional control options reinforce positive perceptions.

Input control has the strongest influence on perceived control.

Abstract

Educational recommender systems (ERSs) are becoming increasingly important in enhancing educational outcomes and personalizing learning experiences by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. While user control is widely assumed to improve user experience, the effects of different levels of control in ERSs remain underexplored. To address this gap, we designed and evaluated an interactive ERS within the MOOC platform CourseMapper, where learners could interact with the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the system. We conducted a between-subjects user study (N=184) to examine how varying levels of user control in an ERS influenced users' perceptions of the recommendation goals of perceived control, transparency, trust, satisfaction, and perceived quality. Our results show that enabling users to build and refine their profile is sufficient to promote positive perceptions of the ERS, while additional control options mainly reinforce these impressions. Moreover, perceived control is the only goal significantly affected by providing different levels of user control in the ERS, with input control exerting the strongest influence. Furthermore, different levels of control affect transparency, trust, satisfaction, and perceived quality in distinct yet interconnected ways. Overall, the findings provide empirical evidence that user control positively shapes transparency, trust, satisfaction, and perceived quality, though to varying extents.