Incentivizing Quality Text Generation via Statistical Contracts

TL;DR

This paper proposes a contract-based economic framework to incentivize high-quality text generation from large language models, addressing moral hazard issues in pay-per-token schemes through cost-robust contracts and statistical hypothesis testing.

Contribution

It introduces a novel economic model with cost-robust contracts for incentivizing quality, extending contract theory with a statistical hypothesis testing approach, and empirically evaluates its effectiveness.

Findings

Cost-robust contracts only marginally reduce objective value.

The framework effectively aligns incentives for high-quality text.

Contracts are adaptable across various evaluation benchmarks.

Abstract

While the success of large language models (LLMs) increases demand for machine-generated text, current pay-per-token pricing schemes create a misalignment of incentives known in economics as moral hazard: Text-generating agents have strong incentive to cut costs by preferring a cheaper model over the cutting-edge one, and this can be done "behind the scenes" since the agent performs inference internally. In this work, we approach this issue from an economic perspective, by proposing a pay-for-performance, contract-based framework for incentivizing quality. We study a principal-agent game where the agent generates text using costly inference, and the contract determines the principal's payment for the text according to an automated quality evaluation. Since standard contract theory is inapplicable when internal inference costs are unknown, we introduce cost-robust contracts. As our main theoretical contribution, we characterize optimal cost-robust contracts through a direct correspondence to optimal composite hypothesis tests from statistics, generalizing a result of Saig et al. (NeurIPS'23). We evaluate our framework empirically by deriving contracts for a range of objectives and LLM evaluation benchmarks, and find that cost-robust contracts sacrifice only a marginal increase in objective value compared to their cost-aware counterparts.