Learning to Trust: Understanding Editorial Authority and Trust in Recommender Systems for Education

TL;DR

This study explores how editorial authority influences trust in educational recommender systems, revealing that stakeholder power dynamics significantly affect trust and system control preferences.

Contribution

It introduces a novel metric of editorial authority based on stakeholder preferences and empirically examines its impact on trust in educational RS.

Findings

Higher editorial authority for students increases trust from course staff.

Course staff prefer greater algorithm control for sourcing and updating recommendations.

Stakeholder roles and rationales reflect perceived expertise and educational needs.

Abstract

Trust in a recommendation system (RS) is often algorithmically incorporated using implicit or explicit feedback of user-perceived trustworthy social neighbors, and evaluated using user-reported trustworthiness of recommended items. However, real-life recommendation settings can feature group disparities in trust, power, and prerogatives. Our study examines a complementary view of trust which relies on the editorial power relationships and attitudes of all stakeholders in the RS application domain. We devise a simple, first-principles metric of editorial authority, i.e., user preferences for recommendation sourcing, veto power, and incorporating user feedback, such that one RS user group confers trust upon another by ceding or assigning editorial authority. In a mixed-methods study at Virginia Tech, we surveyed faculty, teaching assistants, and students about their preferences of editorial authority, and hypothesis-tested its relationship with trust in algorithms for a hypothetical `Suggested Readings' RS. We discover that higher RS editorial authority assigned to students is linked to the relative trust the course staff allocates to RS algorithm and students. We also observe that course staff favors higher control for the RS algorithm in sourcing and updating the recommendations long-term. Using content analysis, we discuss frequent staff-recommended student editorial roles and highlight their frequent rationales, such as perceived expertise, scaling the learning environment, professional curriculum needs, and learner disengagement. We argue that our analyses highlight critical user preferences to help detect editorial power asymmetry and identify RS use-cases for supporting teaching and research