NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation

TL;DR

NineRec introduces a comprehensive benchmark dataset suite for transfer learning in recommendation systems, enabling models to learn from multimodal features across diverse domains and advancing the development of transferable recommendation models.

Contribution

The paper presents NineRec, a large-scale, multimodal dataset suite for transfer learning in recommendation systems, along with benchmark results and insights into model performance.

Findings

Robust benchmark results with classical architectures.

Models can learn from raw multimodal features.

NineRec facilitates transfer learning research in recommendation systems.

Abstract

Large foundational models, through upstream pre-training and downstream fine-tuning, have achieved immense success in the broad AI community due to improved model performance and significant reductions in repetitive engineering. By contrast, the transferable one-for-all models in the recommender system field, referred to as TransRec, have made limited progress. The development of TransRec has encountered multiple challenges, among which the lack of large-scale, high-quality transfer learning recommendation dataset and benchmark suites is one of the biggest obstacles. To this end, we introduce NineRec, a TransRec dataset suite that comprises a large-scale source domain recommendation dataset and nine diverse target domain recommendation datasets. Each item in NineRec is accompanied by a descriptive text and a high-resolution cover image. Leveraging NineRec, we enable the implementation of TransRec models by learning from raw multimodal features instead of relying solely on pre-extracted off-the-shelf features. Finally, we present robust TransRec benchmark results with several classical network architectures, providing valuable insights into the field. To facilitate further research, we will release our code, datasets, benchmarks, and leaderboards at https://github.com/westlake-repl/NineRec.