LLMs for Explainable Business Decision-Making: A Reinforcement Learning Fine-Tuning Approach

TL;DR

This paper introduces LEXMA, a reinforcement learning framework that fine-tunes large language models to generate coherent, audience-specific explanations for business decisions, improving transparency and user understanding without extensive human annotations.

Contribution

The paper presents a novel RL-based fine-tuning method for LLMs that produces multi-audience explanations, addressing key challenges in explainability and label efficiency in business decision contexts.

Findings

LEXMA improves decision accuracy over baseline LLMs.

Generated explanations are more risk-focused and clearer.

Human evaluations favor LEXMA's explanations for different audiences.

Abstract

Artificial Intelligence (AI) models increasingly drive high-stakes consumer interactions, yet their decision logic often remains opaque. Prevailing explainable AI techniques rely on post hoc numerical feature attributions, which fail to provide coherent narratives behind model decisions. Large language models (LLMs) present an opportunity to generate natural-language explanations, but three design challenges remain unresolved: explanations must be both decision-correct and faithful to the factors that drive the prediction; they should be able to serve multiple audiences without shifting the underlying decision rule; and they should be trained in a label-efficient way that does not depend on large corpora of human-scored explanations. To address these challenges, we introduce LEXMA (LLM-based EXplanations for Multi-Audience decisions), a reinforcement-learning-based fine-tuning framework that produces narrative-driven, audience-appropriate explanations. LEXMA combines reflection-augmented supervised fine-tuning with two stages of Group Relative Policy Optimization (GRPO). Specifically, it fine-tunes two separate parameter sets to improve decision correctness and satisfy stylistic requirements for different audiences, using reward signals that do not rely on human-annotated explanations. We instantiate LEXMA in the context of mortgage approval decisions. Results demonstrate that LEXMA yields significant improvements in predictive performance compared with other LLM baselines. Moreover, human evaluations show that expert-facing explanations generated by our approach are more risk-focused, and consumer-facing explanations are clearer, more actionable, and more polite. Our study contributes a cost-efficient, systematic LLM fine-tuning approach to enhance explanation quality for business decisions, offering strong potential for scalable deployment of transparent AI systems.