LLMs for Resource Allocation: A Participatory Budgeting Approach to Inferring Preferences

TL;DR

This paper explores how Large Language Models can be used for resource allocation through participatory budgeting, evaluating their reasoning and preference inference capabilities with various prompting strategies and benchmarks.

Contribution

It introduces a dual-purpose framework using participatory budgeting as a practical task and an adaptive benchmark for assessing LLM reasoning and preference inference.

Findings

LLMs can effectively select project subsets under constraints with proper prompts.

Prompt design significantly impacts LLM performance in resource allocation.

LLMs show potential in inferring structured preferences from natural language inputs.

Abstract

Large Language Models (LLMs) are increasingly expected to handle complex decision-making tasks, yet their ability to perform structured resource allocation remains underexplored. Evaluating their reasoning is also difficult due to data contamination and the static nature of existing benchmarks. We present a dual-purpose framework leveraging Participatory Budgeting (PB) both as (i) a practical setting for LLM-based resource allocation and (ii) an adaptive benchmark for evaluating their reasoning capabilities. We task LLMs with selecting project subsets under feasibility (e.g., budget) constraints via three prompting strategies: greedy selection, direct optimization, and a hill-climbing-inspired refinement. We benchmark LLMs' allocations against a utility-maximizing oracle. Interestingly, we also test whether LLMs can infer structured preferences from natural-language voter input or metadata, without explicit votes. By comparing allocations based on inferred preferences to those from ground-truth votes, we evaluate LLMs' ability to extract preferences from open-ended input. Our results underscore the role of prompt design and show that LLMs hold promise for mechanism design with unstructured inputs.