Interactive Learning with Pricing for Optimal and Stable Allocations in Markets

TL;DR

This paper introduces an interactive learning framework for online recommendation markets that balances maximizing social welfare with ensuring stable, incentive-compatible allocations by integrating market pricing and preference learning.

Contribution

It presents a novel algorithm combining combinatorial bandits, resource allocation, and pricing strategies to achieve low regret in both social welfare and market stability.

Findings

Achieves sub-linear regret in social welfare and stability.

Demonstrates superior performance on synthetic and real data.

First to integrate market pricing with preference learning in this context.

Abstract

Large-scale online recommendation systems must facilitate the allocation of a limited number of items among competing users while learning their preferences from user feedback. As a principled way of incorporating market constraints and user incentives in the design, we consider our objectives to be two-fold: maximal social welfare with minimal instability. To maximize social welfare, our proposed framework enhances the quality of recommendations by exploring allocations that optimistically maximize the rewards. To minimize instability, a measure of users' incentives to deviate from recommended allocations, the algorithm prices the items based on a scheme derived from the Walrasian equilibria. Though it is known that these equilibria yield stable prices for markets with known user preferences, our approach accounts for the inherent uncertainty in the preferences and further ensures that the users accept their recommendations under offered prices. To the best of our knowledge, our approach is the first to integrate techniques from combinatorial bandits, optimal resource allocation, and collaborative filtering to obtain an algorithm that achieves sub-linear social welfare regret as well as sub-linear instability. Empirical studies on synthetic and real-world data also demonstrate the efficacy of our strategy compared to approaches that do not fully incorporate all these aspects.