Revisiting Reciprocal Recommender Systems: Metrics, Formulation, and Method

TL;DR

This paper introduces new holistic metrics, a causal formulation, and a model-agnostic method for reciprocal recommender systems, improving evaluation and matching effectiveness in bilateral recommendation scenarios.

Contribution

It proposes five new evaluation metrics, a causal formulation of RRS, and a reranking strategy, advancing holistic assessment and causal modeling in reciprocal recommendation.

Findings

New metrics improve holistic evaluation of RRS

Causal formulation enhances recommendation modeling

Reranking strategy boosts matching outcomes

Abstract

Reciprocal recommender systems~(RRS), conducting bilateral recommendations between two involved parties, have gained increasing attention for enhancing matching efficiency. However, the majority of existing methods in the literature still reuse conventional ranking metrics to separately assess the performance on each side of the recommendation process. These methods overlook the fact that the ranking outcomes of both sides collectively influence the effectiveness of the RRS, neglecting the necessity of a more holistic evaluation and a capable systemic solution. In this paper, we systemically revisit the task of reciprocal recommendation, by introducing the new metrics, formulation, and method. Firstly, we propose five new evaluation metrics that comprehensively and accurately assess the performance of RRS from three distinct perspectives: overall coverage, bilateral stability, and balanced ranking. These metrics provide a more holistic understanding of the system's effectiveness and enable a comprehensive evaluation. Furthermore, we formulate the RRS from a causal perspective, formulating recommendations as bilateral interventions, which can better model the decoupled effects of potential influencing factors. By utilizing the potential outcome framework, we further develop a model-agnostic causal reciprocal recommendation method that considers the causal effects of recommendations. Additionally, we introduce a reranking strategy to maximize matching outcomes, as measured by the proposed metrics. Extensive experiments on two real-world datasets from recruitment and dating scenarios demonstrate the effectiveness of our proposed metrics and approach. The code and dataset are available at: https://github.com/RUCAIBox/CRRS.