RecNextEval: A Reference Implementation for Temporal Next-Batch Recommendation Evaluation

TL;DR

RecNextEval is an open-source evaluation framework for next-batch recommendation that uses time-window data splits to improve realism and reduce data leakage in RecSys research.

Contribution

It introduces a reference implementation with a GUI for more accurate, timeline-aware evaluation of recommendation models, addressing limitations of existing protocols.

Findings

Utilizes time-window data split to evaluate models along a global timeline.

Highlights complexities in RecSys evaluation and promotes realistic model development.

Provides an open-source library with GUI for reproducible, fair evaluation.

Abstract

A good number of toolkits have been developed in Recommender Systems (RecSys) research to promote fair evaluation and reproducibility. However, recent critical examinations of RecSys evaluation protocols have raised concerns regarding the validity of existing evaluation pipelines. In this demonstration, we present RecNextEval, a reference implementation of an evaluation framework specifically designed for next-batch recommendation. RecNextEval utilizes a time-window data split to ensure models are evaluated along a global timeline, effectively minimizing data leakage. Our implementation highlights the inherent complexities of RecSys evaluation and encourages a shift toward model development that more accurately simulates production environments. The RecNextEval library and its accompanying GUI interface are open-source and publicly accessible.