Carrot and Stick: Eliciting Comparison Data and Beyond
Yiling Chen, Shi Feng, Fang-Yi Yu

TL;DR
This paper introduces peer prediction mechanisms with bonus-penalty payments to truthfully elicit comparison data from individuals, applicable to machine learning tasks like ranking and reinforcement learning from human feedback, supported by theoretical analysis and real-world experiments.
Contribution
It develops symmetrically strongly truthful peer prediction mechanisms leveraging stochastic transitivity, extending to networked data with Ising model signals, and provides theoretical conditions and experimental validation.
Findings
Mechanisms ensure truth-telling as a strict Bayesian Nash equilibrium.
Designed mechanisms require only one comparison evaluation per individual.
Experimental results support theoretical claims on real-world datasets.
Abstract
Comparison data elicited from people are fundamental to many machine learning tasks, including reinforcement learning from human feedback for large language models and estimating ranking models. They are typically subjective and not directly verifiable. How to truthfully elicit such comparison data from rational individuals? We design peer prediction mechanisms for eliciting comparison data using a bonus-penalty payment. Our design leverages on the strong stochastic transitivity for comparison data to create symmetrically strongly truthful mechanisms such that truth-telling 1) forms a strict Bayesian Nash equilibrium, and 2) yields the highest payment among all symmetric equilibria. Each individual only needs to evaluate one pair of items and report her comparison in our mechanism. We further extend the bonus-penalty payment concept to eliciting networked data, designing a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsMobile Crowdsensing and Crowdsourcing · Ethics and Social Impacts of AI · Explainable Artificial Intelligence (XAI)
