Can Large Language Models Capture Human Risk Preferences? A Cross-Cultural Study

TL;DR

This study evaluates how well large language models can simulate human risk preferences across different cultures, revealing their current limitations and the influence of language on their decision-making accuracy.

Contribution

It provides a cross-cultural comparison of LLMs' ability to predict human risky decisions, highlighting their risk aversion and language-dependent performance.

Findings

LLMs are more risk-averse than humans in lottery tasks.

Model predictions align better with humans in English than in Chinese.

Language and cultural context affect LLMs' decision-making accuracy.

Abstract

Large language models (LLMs) have made significant strides, extending their applications to dialogue systems, automated content creation, and domain-specific advisory tasks. However, as their use grows, concerns have emerged regarding their reliability in simulating complex decision-making behavior, such as risky decision-making, where a single choice can lead to multiple outcomes. This study investigates the ability of LLMs to simulate risky decision-making scenarios. We compare model-generated decisions with actual human responses in a series of lottery-based tasks, using transportation stated preference survey data from participants in Sydney, Dhaka, Hong Kong, and Nanjing. Demographic inputs were provided to two LLMs -- ChatGPT 4o and ChatGPT o1-mini -- which were tasked with predicting individual choices. Risk preferences were analyzed using the Constant Relative Risk Aversion (CRRA) framework. Results show that both models exhibit more risk-averse behavior than human participants, with o1-mini aligning more closely with observed human decisions. Further analysis of multilingual data from Nanjing and Hong Kong indicates that model predictions in Chinese deviate more from actual responses compared to English, suggesting that prompt language may influence simulation performance. These findings highlight both the promise and the current limitations of LLMs in replicating human-like risk behavior, particularly in linguistic and cultural settings.