Truthful Reporting of Competence with Minimal Verification

TL;DR

This paper explores designing mechanisms for truthful self-reporting of competence with limited verification, balancing bias minimization and truthful incentives, including cases with perfect and noisy verification.

Contribution

It characterizes the optimal tradeoff between verification effort and bias reduction in truthful reporting mechanisms, providing a simple optimal mechanism for perfect verification and approaches for noisy cases.

Findings

Optimal verification strategies depend on agents' type distribution.

Proper scoring rules can improve truthfulness under noisy verification.

A simple parametrized mechanism achieves optimal bias-verification tradeoff with perfect verification.

Abstract

Suppose you run a home exam, where students should report their own scores but can cheat freely. You can, if needed, call a limited number of students to class and verify their actual performance against their reported score. We consider the class of mechanisms where truthful reporting is a dominant strategy, and truthful agents are never penalized -- even off-equilibrium. How many students do we need to verify, in expectation, if we want to minimize the bias, i.e., the difference between agents' competence and their expected grade? When perfect verification is available, we characterize the best possible tradeoff between these requirements and provide a simple parametrized mechanism that is optimal in the class for any distribution of agents' types. When verification is noisy, the task becomes much more challenging. We show how proper scoring rules can be leveraged in different ways to construct truthful mechanisms with a good (though not necessarily optimal) tradeoff.