Learning from a Learning User for Optimal Recommendations

TL;DR

This paper models the interaction between learning users and recommendation systems, proposing a new algorithm RAES that adapts to non-stationary feedback, with theoretical and experimental validation.

Contribution

It introduces a formal model for learning users in recommendation systems and develops RAES, an algorithm robust to non-stationary user feedback.

Findings

RAES's regret increases gracefully with slower user learning convergence.

RAES achieves near-optimal performance in synthetic experiments.

The model captures the feedback loop between user learning and system recommendations.

Abstract

In real-world recommendation problems, especially those with a formidably large item space, users have to gradually learn to estimate the utility of any fresh recommendations from their experience about previously consumed items. This in turn affects their interaction dynamics with the system and can invalidate previous algorithms built on the omniscient user assumption. In this paper, we formalize a model to capture such "learning users" and design an efficient system-side learning solution, coined Noise-Robust Active Ellipsoid Search (RAES), to confront the challenges brought by the non-stationary feedback from such a learning user. Interestingly, we prove that the regret of RAES deteriorates gracefully as the convergence rate of user learning becomes worse, until reaching linear regret when the user's learning fails to converge. Experiments on synthetic datasets demonstrate the strength of RAES for such a contemporaneous system-user learning problem. Our study provides a novel perspective on modeling the feedback loop in recommendation problems.