Truthful Aggregation of LLMs with an Application to Online Advertising

TL;DR

This paper introduces MOSAIC, a truthful auction mechanism for aggregating preferences in LLM-generated content, optimizing advertiser and platform utility without requiring LLM fine-tuning, applicable broadly beyond advertising.

Contribution

We propose MOSAIC, a novel auction mechanism that ensures truthful preference reporting and converges to optimal LLM outputs without model fine-tuning, applicable in various preference aggregation scenarios.

Findings

MOSAIC achieves high advertiser value and platform revenue.

The mechanism converges to the optimal fine-tuned LLM output as resources increase.

Incorporating contextual information improves social welfare.

Abstract

The next frontier of online advertising is revenue generation from LLM-generated content. We consider a setting where advertisers aim to influence the responses of an LLM to align with their interests, while platforms seek to maximize advertiser value and ensure user satisfaction. The challenge is that advertisers' preferences generally conflict with those of the user, and advertisers may misreport their preferences. To address this, we introduce MOSAIC, an auction mechanism that ensures that truthful reporting is a dominant strategy for advertisers and that aligns the utility of each advertiser with their contribution to social welfare. Importantly, the mechanism operates without LLM fine-tuning or access to model weights and provably converges to the output of the optimally fine-tuned LLM as computational resources increase. Additionally, it can incorporate contextual information about advertisers, which significantly improves social welfare. Through experiments with a publicly available LLM, we show that MOSAIC leads to high advertiser value and platform revenue with low computational overhead. While our motivating application is online advertising, our mechanism can be applied in any setting with monetary transfers, making it a general-purpose solution for truthfully aggregating the preferences of self-interested agents over LLM-generated replies.