Designing and Evaluating an Educational Recommender System with Different Levels of User Control

TL;DR

This paper systematically designs and evaluates an educational recommender system with varying levels of user control, demonstrating that increased control enhances transparency, trust, and satisfaction among users.

Contribution

It introduces multiple levels of user control in ERS design and empirically evaluates their impact on user perceptions and recommendation goals.

Findings

User control positively affects perceived benefits of ERS.

Transparency strongly correlates with trust and satisfaction.

Trust has a strong correlation with user satisfaction.

Abstract

Educational recommender systems (ERSs) play a crucial role in personalizing learning experiences and enhancing educational outcomes by providing recommendations of personalized resources and activities to learners, tailored to their individual learning needs. However, their effectiveness is often diminished by insufficient user control and limited transparency. To address these challenges, in this paper, we present the systematic design and evaluation of an interactive ERS, in which we introduce different levels of user control. Concretely, we introduce user control around the input (i.e., user profile), process (i.e., recommendation algorithm), and output (i.e., recommendations) of the ERS. To evaluate our system, we conducted an online user study (N=30) to explore the impact of user control on users' perceptions of the ERS in terms of several important user-centric aspects. Moreover, we investigated the effects of user control on multiple recommendation goals, namely transparency, trust, and satisfaction, as well as the interactions between these goals. Our results demonstrate the positive impact of user control on user perceived benefits of the ERS. Moreover, our study shows that user control strongly correlates with transparency and moderately correlates with trust and satisfaction. In terms of interaction between these goals, our results reveal that transparency moderately correlates and trust strongly correlates with satisfaction. Whereas, transparency and trust stand out as less correlated with each other.