LLM-Powered Preference Elicitation in Combinatorial Assignment

TL;DR

This paper explores using large language models as efficient proxies for humans in preference elicitation for combinatorial assignment, reducing effort and improving efficiency.

Contribution

It introduces a framework for integrating LLMs with existing preference elicitation schemes, addressing challenges like response variability and computational costs.

Findings

LLM proxies improve allocative efficiency by up to 20%

Results are consistent across different LLMs and report qualities

Framework effectively combines LLMs with preference elicitation methods

Abstract

We study the potential of large language models (LLMs) as proxies for humans to simplify preference elicitation (PE) in combinatorial assignment. While traditional PE methods rely on iterative queries to capture preferences, LLMs offer a one-shot alternative with reduced human effort. We propose a framework for LLM proxies that can work in tandem with SOTA ML-powered preference elicitation schemes. Our framework handles the novel challenges introduced by LLMs, such as response variability and increased computational costs. We experimentally evaluate the efficiency of LLM proxies against human queries in the well-studied course allocation domain, and we investigate the model capabilities required for success. We find that our approach improves allocative efficiency by up to 20%, and these results are robust across different LLMs and to differences in quality and accuracy of reporting.