No-Regret Online Prediction with Strategic Experts

TL;DR

This paper develops algorithms for online expert prediction that are both incentive-compatible and achieve low regret, even when experts act strategically and report beliefs strategically, extending previous work from the single-expert case to multiple experts.

Contribution

It introduces novel algorithms for the multi-expert setting that ensure truthful reporting and low regret, overcoming inefficiencies of naive reductions from the single-expert case.

Findings

Algorithms achieve sublinear regret with strategic experts.

The proposed methods outperform naive reductions in efficiency and effectiveness.

Applications include forecasting competitions with strategic forecasters.

Abstract

We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick $m\geq 1$ experts from a pool of $K$ experts and the overall utility is a modular or submodular function of the chosen experts. We focus on the setting in which experts act strategically and aim to maximize their influence on the algorithm's predictions by potentially misreporting their beliefs about the events. Among others, this setting finds applications in forecasting competitions where the learner seeks not only to make predictions by aggregating different forecasters but also to rank them according to their relative performance. Our goal is to design algorithms that satisfy the following two requirements: 1) $\textit{Incentive-compatible}$: Incentivize the experts to report their beliefs truthfully, and 2) $\textit{No-regret}$: Achieve sublinear regret with respect to the true beliefs of the best fixed set of $m$ experts in hindsight. Prior works have studied this framework when $m=1$ and provided incentive-compatible no-regret algorithms for the problem. We first show that a simple reduction of our problem to the $m=1$ setting is neither efficient nor effective. Then, we provide algorithms that utilize the specific structure of the utility functions to achieve the two desired goals.