Incentive-Compatible Classification

TL;DR

This paper explores the design of incentive-compatible mechanisms for classifying agents based on peer reviews, establishing bounds on their effectiveness and proposing mechanisms that perform well under certain review constraints.

Contribution

It provides theoretical bounds on the performance of strategy-proof classification mechanisms and introduces a simple mechanism that achieves optimality when review counts are sublinear.

Findings

Bounds on the coincidence between mechanism and ideal classification depend on review limits.

When review counts are unbounded, the trivial mechanism is optimal.

A simple mechanism achieves optimal coincidence ratio when review counts are sublinear.

Abstract

We investigate the possibility of an incentive-compatible (IC, a.k.a. strategy-proof) mechanism for the classification of agents in a network according to their reviews of each other. In the $ \alpha $-classification problem we are interested in selecting the top $ \alpha $ fraction of users. We give upper bounds (impossibilities) and lower bounds (mechanisms) on the worst-case coincidence between the classification of an IC mechanism and the ideal $ \alpha $-classification. We prove bounds which depend on $ \alpha $ and on the maximal number of reviews given by a single agent, $ \Delta $. Our results show that it is harder to find a good mechanism when $ \alpha $ is smaller and $ \Delta $ is larger. In particular, if $ \Delta $ is unbounded, then the best mechanism is trivial (that is, it does not take into account the reviews). On the other hand, when $ \Delta $ is sublinear in the number of agents, we give a simple, natural mechanism, with a coincidence ratio of $ \alpha $.