DecisionFlow: Advancing Large Language Model as Principled Decision Maker

TL;DR

DecisionFlow introduces a structured decision modeling framework for large language models, enabling transparent reasoning and improved accuracy in high-stakes decision-making tasks.

Contribution

It presents a novel approach that guides LLMs to reason over structured representations and infer utility functions, enhancing explainability and performance.

Findings

Achieves up to 30% accuracy improvement over prompting baselines.

Enhances alignment and interpretability of decisions.

Demonstrates effectiveness on high-stakes benchmarks.

Abstract

In high-stakes domains such as healthcare and finance, effective decision-making demands not just accurate outcomes but transparent and explainable reasoning. However, current language models often lack the structured deliberation needed for such tasks, instead generating decisions and justifications in a disconnected, post-hoc manner. To address this, we propose DecisionFlow, a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints. Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function to evaluate trade-offs in a transparent, utility-driven manner. This process produces decisions tightly coupled with interpretable rationales reflecting the model's reasoning. Empirical results on two high-stakes benchmarks show that DecisionFlow not only achieves up to 30% accuracy gains over strong prompting baselines but also enhances alignment in outcomes. Our work is a critical step toward integrating symbolic reasoning with LLMs, enabling more accountable, explainable, and reliable LLM decision support systems. Code and data are at https://github.com/xiusic/DecisionFlow.