Benchmarking Deep Neural Networks for Modern Recommendation Systems

TL;DR

This paper benchmarks seven deep neural network architectures for recommendation systems across multiple real-world datasets, evaluating them on accuracy, diversity, relational awareness, temporal dynamics, and efficiency to guide model selection and combination.

Contribution

It introduces a Requirement-Oriented Benchmarking framework for fair, comprehensive evaluation of neural architectures tailored to modern recommendation system demands.

Findings

Different architectures excel in different requirements.

Hybrid and ensemble models are motivated by complementary strengths.

Practical guidance for model selection and combination is provided.

Abstract

This paper presents a requirement-oriented benchmark of seven deep neural architectures, CNN, RNN, GNN, Autoencoder, Transformer, Neural Collaborative Filtering, and Siamese Networks, across three real-world datasets: Retail E-commerce, Amazon Products, and Netflix Prize. To ensure a fair and comprehensive comparison aligned with the evolving demands of modern recommendation systems, we adopt a Requirement-Oriented Benchmarking (ROB) framework that structures evaluation around predictive accuracy, recommendation diversity, relational awareness, temporal dynamics, and computational efficiency. Under a unified evaluation protocol, models are assessed using standard accuracy-oriented metrics alongside diversity and efficiency indicators. Experimental results show that different architectures exhibit complementary strengths across requirements, motivating the use of hybrid and ensemble designs. The findings provide practical guidance for selecting and combining neural architectures to better satisfy multi-objective recommendation system requirements.