Look-Ahead Reasoning on Learning Platforms

TL;DR

This paper explores how users strategically interact with learning platforms by looking ahead and coordinating, analyzing the effects on model outcomes and utility alignment, with implications for designing better learning systems.

Contribution

It formalizes level-k and collective reasoning in learning platforms, analyzing their impact on convergence, equilibrium, and utility alignment, introducing new insights into strategic user behavior.

Findings

Higher-level reasoning accelerates convergence but does not change the equilibrium.

Coordination among users can improve collective utility and alignment.

The paper characterizes utility trade-offs in collective action against algorithms.

Abstract

On many learning platforms, the optimization criteria guiding model training reflect the priorities of the designer rather than those of the individuals they affect. Consequently, users may act strategically to obtain more favorable outcomes. While past work has studied strategic user behavior on learning platforms, the focus has largely been on strategic responses to a deployed model, without considering the behavior of other users. In contrast, look-ahead reasoning takes into account that user actions are coupled, and -- at scale -- impact future predictions. Within this framework, we first formalize level-k thinking, a concept from behavioral economics, where users aim to outsmart their peers by looking one step ahead. We show that, while convergence to an equilibrium is accelerated, the equilibrium remains the same, providing no benefit of higher-level reasoning for individuals in the long run. Then, we focus on collective reasoning, where users take coordinated actions by optimizing through their joint impact on the model. By contrasting collective with selfish behavior, we characterize the benefits and limits of coordination; a new notion of alignment between the learner's and the users' utilities emerges as a key concept. Look-ahead reasoning can be seen as a generalization of algorithmic collective action; we thus offer the first results characterizing the utility trade-offs of coordination when contesting algorithmic systems.