Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions

TL;DR

This paper investigates using reinforcement learning to fine-tune large language models so they can both predict human decision-making and generate interpretable natural language explanations of cognitive processes.

Contribution

It introduces a reinforcement learning approach to train LLMs for dual tasks of prediction and explanation of human decisions, enhancing interpretability.

Findings

LLMs can generate high-quality explanations of human risky choices.

Reinforcement learning improves the alignment of explanations with decision predictions.

The method achieves strong predictive performance and interpretability in cognitive modeling.

Abstract

A central goal of cognitive modeling is to develop models that not only predict human behavior but also provide insight into the underlying cognitive mechanisms. While neural network models trained on large-scale behavioral data often achieve strong predictive performance, they typically fall short in offering interpretable explanations of the cognitive processes they capture. In this work, we explore the potential of pretrained large language models (LLMs) to serve as dual-purpose cognitive models--capable of both accurate prediction and interpretable explanation in natural language. Specifically, we employ reinforcement learning with outcome-based rewards to guide LLMs toward generating explicit reasoning traces for explaining human risky choices. Our findings demonstrate that this approach produces high-quality explanations alongside strong quantitative predictions of human decisions.