Extrapolating Volition with Recursive Information Markets

TL;DR

This paper introduces a recursive information market mechanism utilizing LLM buyers to address information asymmetry, with implications for AI alignment and scalable oversight.

Contribution

It formalizes a recursive mechanism for information markets using LLMs, analyzing its incentive properties and potential applications in AI alignment.

Findings

The recursive mechanism incentivizes truthful information pricing.

It overcomes the buyer's inspection paradox in information markets.

Potential applications include AI alignment and scalable oversight.

Abstract

One of the impediments to the efficiency of information markets is the inherent information asymmetry present in them, exacerbated by the "buyer's inspection paradox" (the buyer cannot mitigate the asymmetry by "inspecting" the information, because in doing so the buyer obtains the information without paying for it). Previous work has suggested that using Large Language Model (LLM) buyers to inspect and purchase information could overcome this information asymmetry, as an LLM buyer can simply "forget" the information it inspects. In this work, we analyze this mechanism formally through a "value-of-information" paradigm, i.e. whether it incentivizes information to be priced and provided in accordance with its "true value". We focus in particular on our new recursive version of the mechanism, which we believe has a range of applications including in AI alignment research, where it is related to Extrapolated Volition and Scalable Oversight.