LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection

TL;DR

LISTEN introduces an LLM-based framework that iteratively refines preferences and makes decisions for multi-objective tasks, reducing the need for explicit preference formalization.

Contribution

The paper proposes two novel algorithms, LISTEN-U and LISTEN-T, for multi-objective decision-making using LLMs, with a new metric for preference alignment.

Findings

LISTEN-U performs well with parametrically aligned preferences.

LISTEN-T provides robust performance across diverse tasks.

The framework reduces the cognitive burden of preference elicitation.

Abstract

Human experts often struggle to select the best option from a large set of items with multiple competing objectives, a process bottlenecked by the difficulty of formalizing complex, implicit preferences. To address this, we introduce LISTEN (LLM-based Iterative Selection with Trade-off Evaluation from Natural-language), an agentic LLM-based framework that treats the LLM as a decision-making agent capable of iteratively refining its internal preference model and taking actions (e.g., proposing utilities or selecting candidates) to maximize alignment with a user's implicit goals. To operate within LLM constraints like context windows and inference costs, we propose two iterative algorithms: LISTEN-U, which uses the LLM to refine a parametric utility function, and LISTEN-T, a non-parametric method that performs tournament-style selections over small batches of solutions. Evaluated on diverse tasks including flight booking, shopping, and exam scheduling, our results show LISTEN-U excels when preferences are parametrically aligned (a property we measure with a novel concordance metric), while LISTEN-T offers more robust performance overall. This work explores a promising direction for steering complex multi-objective decisions directly with natural language, reducing the cognitive burden of traditional preference elicitation. Code is available at https://github.com/AdamJovine/LISTEN.