Knowledge is Power, Understanding is Impact: Utility and Beyond Goals, Explanation Quality, and Fairness in Path Reasoning Recommendation

TL;DR

This paper evaluates state-of-the-art path reasoning recommendation methods using a unified protocol, analyzing their utility, explanation quality, and fairness to identify progress and open issues in the field.

Contribution

It replicates and assesses three leading path reasoning methods under a common evaluation framework, providing a comprehensive comparison and insights into their strengths and limitations.

Findings

Path reasoning methods improve recommendation utility but vary in explanation quality.

Evaluation reveals gaps in fairness and transparency among current methods.

The study highlights open challenges and suggests future research directions.

Abstract

Path reasoning is a notable recommendation approach that models high-order user-product relations, based on a Knowledge Graph (KG). This approach can extract reasoning paths between recommended products and already experienced products and, then, turn such paths into textual explanations for the user. Unfortunately, evaluation protocols in this field appear heterogeneous and limited, making it hard to contextualize the impact of the existing methods. In this paper, we replicated three state-of-the-art relevant path reasoning recommendation methods proposed in top-tier conferences. Under a common evaluation protocol, based on two public data sets and in comparison with other knowledge-aware methods, we then studied the extent to which they meet recommendation utility and beyond objectives, explanation quality, and consumer and provider fairness. Our study provides a picture of the progress in this field, highlighting open issues and future directions. Source code: \url{https://github.com/giacoballoccu/rep-path-reasoning-recsys}.