Practitioners Versus Users: A Value-Sensitive Evaluation of Current Industrial Recommender System Design

TL;DR

This paper investigates how practitioners and users perceive and prioritize values like privacy, transparency, and fairness in industrial recommender systems, highlighting tensions and implications for human-centric design.

Contribution

It introduces a comprehensive value-sensitive evaluation framework for industrial recommender systems, combining conceptual and empirical insights from practitioners and users.

Findings

Identifies value tensions between practitioners and users.

Highlights sources of value conflicts in recommender system design.

Provides implications for human-centric recommender system development.

Abstract

Recommender systems are playing an increasingly important role in alleviating information overload and supporting users' various needs, e.g., consumption, socialization, and entertainment. However, limited research focuses on how values should be extensively considered in industrial deployments of recommender systems, the ignorance of which can be problematic. To fill this gap, in this paper, we adopt Value Sensitive Design to comprehensively explore how practitioners and users recognize different values of current industrial recommender systems. Based on conceptual and empirical investigations, we focus on five values: recommendation quality, privacy, transparency, fairness, and trustworthiness. We further conduct in-depth qualitative interviews with 20 users and 10 practitioners to delve into their opinions about these values. Our results reveal the existence and sources of tensions between practitioners and users in terms of value interpretation, evaluation, and practice, which provide novel implications for designing more human-centric and value-sensitive recommender systems.