Learning to Recommend in Unknown Games

TL;DR

This paper investigates how a moderator can learn agents' utility functions in multi-agent games through recommendations, analyzing different feedback models and proposing algorithms with low regret for strategic environments.

Contribution

It provides a theoretical analysis of learnability under different feedback models and introduces an online algorithm with low regret for strategic recommendation scenarios.

Findings

Quantal-response feedback allows logarithmic sample complexity learning.

Best-response feedback identifies a larger set of utilities, less precisely.

The proposed algorithm achieves low regret with bounds depending on game dimension and time.

Abstract

We study preference learning through recommendations in multi-agent game settings, where a moderator repeatedly interacts with agents whose utility functions are unknown. In each round, the moderator issues action recommendations and observes whether agents follow or deviate from them. We consider two canonical behavioral feedback models-best response and quantal response-and study how the information revealed by each model affects the learnability of agents' utilities. We show that under quantal-response feedback the game is learnable, up to a positive affine equivalence class, with logarithmic sample complexity in the desired precision, whereas best-response feedback can only identify a larger set of agents' utilities. We give a complete geometric characterization of this set. Moreover, we introduce a regret notion based on agents' incentives to deviate from recommendations and design an online algorithm with low regret under both feedback models, with bounds scaling linearly in the game dimension and logarithmically in time. Our results lay a theoretical foundation for AI recommendation systems in strategic multi-agent environments, where recommendation compliances are shaped by strategic interaction.