A Reproducible and Fair Evaluation of Partition-aware Collaborative Filtering

TL;DR

This paper provides a transparent, reproducible benchmark of partition-aware collaborative filtering models, revealing their strengths, limitations, and optimal scenarios for use in scalable recommender systems.

Contribution

It introduces a fully reproducible evaluation framework for FPSR models and clarifies their performance dynamics and practical benefits.

Findings

FPSR models are not always the top performers but are competitive.

Partitioning benefits are most evident in long-tail recommendation scenarios.

The study offers guidance for designing scalable recommender systems with partition-aware methods.

Abstract

Similarity-based collaborative filtering (CF) models have long demonstrated strong offline performance and conceptual simplicity. However, their scalability is limited by the quadratic cost of maintaining dense item-item similarity matrices. Partitioning-based paradigms have recently emerged as an effective strategy for balancing effectiveness and efficiency, enabling models to learn local similarities within coherent subgraphs while maintaining a limited global context. In this work, we focus on the Fine-tuning Partition-aware Similarity Refinement (FPSR) framework, a prominent representative of this family, as well as its extension, FPSR+. Reproducible evaluation of partition-aware collaborative filtering remains challenging, as prior FPSR/FPSR+ reports often rely on splits of unclear provenance and omit some similarity-based baselines, thereby complicating fair comparison. We present a transparent, fully reproducible benchmark of FPSR and FPSR+. Based on our results, the family of FPSR models does not consistently perform at the highest level. Overall, it remains competitive, validates its design choices, and shows significant advantages in long-tail scenarios. This highlights the accuracy-coverage trade-offs resulting from partitioning, global components, and hub design. Our investigation clarifies when partition-aware similarity modeling is most beneficial and offers actionable guidance for scalable recommender system design under reproducible protocols.