Fuzzy synthetic method for evaluating explanations in recommender systems

TL;DR

This paper investigates the impact of context-aware explanations in recommender systems through a user study and introduces a fuzzy synthetic evaluation method to aggregate various user experience metrics.

Contribution

It presents a novel fuzzy synthetic evaluation approach for aggregating explanation effectiveness metrics and provides empirical insights from a user study on context-aware explanations.

Findings

Context-aware explanations improve user trust and satisfaction.

Fuzzy synthetic evaluation effectively combines multiple metrics.

User study results highlight the importance of explanation transparency.

Abstract

Recommender systems aim to help users find relevant items more quickly by providing personalized recommendations. Explanations in recommender systems help users understand why such recommendations have been generated, which in turn makes the system more transparent and promotes users' trust and satisfaction. In recent years, explaining recommendations has drawn increasing attention from both academia and from industry. In this paper, we present a user study to investigate context-aware explanations in recommender systems. In particular, we build a web-based questionnaire that is able to interact with users: generating and explaining recommendations. With this questionnaire, we investigate the effects of context-aware explanations in terms of efficiency, effectiveness, persuasiveness, satisfaction, trust and transparency through a user study. Besides, we propose a novel method based on fuzzy synthetic evaluation for aggregating these metrics.