Epsilon non-Greedy: A Bandit Approach for Unbiased Recommendation via Uniform Data

TL;DR

This paper introduces a bandit-based framework for unbiased recommendation systems that mitigates self-feedback bias by leveraging uniform data collection and sequential training, improving over existing debiasing methods.

Contribution

It proposes a novel offline sequential training schema and a bandit approach to reduce bias in recommendation systems, addressing the feedback loop issue.

Findings

Outperforms state-of-the-art debiasing methods in experiments

Effectively explores under-understood items to improve recommendations

Simulates real-world continuous training scenarios

Abstract

Often, recommendation systems employ continuous training, leading to a self-feedback loop bias in which the system becomes biased toward its previous recommendations. Recent studies have attempted to mitigate this bias by collecting small amounts of unbiased data. While these studies have successfully developed less biased models, they ignore the crucial fact that the recommendations generated by the model serve as the training data for subsequent training sessions. To address this issue, we propose a framework that learns an unbiased estimator using a small amount of uniformly collected data and focuses on generating improved training data for subsequent training iterations. To accomplish this, we view recommendation as a contextual multi-arm bandit problem and emphasize on exploring items that the model has a limited understanding of. We introduce a new offline sequential training schema that simulates real-world continuous training scenarios in recommendation systems, offering a more appropriate framework for studying self-feedback bias. We demonstrate the superiority of our model over state-of-the-art debiasing methods by conducting extensive experiments using the proposed training schema.