Decisive: Guiding User Decisions with Optimal Preference Elicitation from Unstructured Documents

TL;DR

Decisive introduces an interactive framework that combines document-grounded reasoning with Bayesian preference inference to improve decision-making efficiency and personalization, outperforming existing methods.

Contribution

It presents a novel approach that actively learns user preferences through targeted questions, integrating unstructured document analysis with Bayesian inference for better decision support.

Findings

Achieves up to 20% improvement in decision accuracy over baselines.

Efficiently converges with minimal user effort.

Outperforms general-purpose LLMs in decision tasks.

Abstract

Decision-making is a cognitively intensive task that requires synthesizing relevant information from multiple unstructured sources, weighing competing factors, and incorporating subjective user preferences. Existing methods, including large language models and traditional decision-support systems, fall short: they often overwhelm users with information or fail to capture nuanced preferences accurately. We present Decisive, an interactive decision-making framework that combines document-grounded reasoning with Bayesian preference inference. Our approach grounds decisions in an objective option-scoring matrix extracted from source documents, while actively learning a user's latent preference vector through targeted elicitation. Users answer pairwise tradeoff questions adaptively selected to maximize information gain over the final decision. This process converges efficiently, minimizing user effort while ensuring recommendations remain transparent and personalized. Through extensive experiments, we demonstrate that our approach significantly outperforms both general-purpose LLMs and existing decision-making frameworks achieving up to 20% improvement in decision accuracy over strong baselines across domains.