Gradient-based Optimization for Bayesian Preference Elicitation

TL;DR

This paper introduces a scalable, gradient-based Bayesian preference elicitation method that efficiently optimizes queries in large item spaces, improving recommendation systems' interactivity and effectiveness.

Contribution

It presents a novel differentiable formulation of EVOI and a Monte Carlo optimization approach that scales to large item spaces and diverse query types.

Findings

Achieves state-of-the-art performance in preference elicitation tasks.

Scales efficiently to large item spaces using gradient-based optimization.

Adapts to various query formats, including partial items.

Abstract

Effective techniques for eliciting user preferences have taken on added importance as recommender systems (RSs) become increasingly interactive and conversational. A common and conceptually appealing Bayesian criterion for selecting queries is expected value of information (EVOI). Unfortunately, it is computationally prohibitive to construct queries with maximum EVOI in RSs with large item spaces. We tackle this issue by introducing a continuous formulation of EVOI as a differentiable network that can be optimized using gradient methods available in modern machine learning (ML) computational frameworks (e.g., TensorFlow, PyTorch). We exploit this to develop a novel, scalable Monte Carlo method for EVOI optimization, which is more scalable for large item spaces than methods requiring explicit enumeration of items. While we emphasize the use of this approach for pairwise (or k-wise) comparisons of items, we also demonstrate how our method can be adapted to queries involving subsets of item attributes or "partial items," which are often more cognitively manageable for users. Experiments show that our gradient-based EVOI technique achieves state-of-the-art performance across several domains while scaling to large item spaces.