Predicting and Understanding Human Action Decisions: Insights from Large Language Models and Cognitive Instance-Based Learning

TL;DR

This paper explores how Large Language Models and cognitive models can predict and understand human decision-making, revealing their respective strengths in feedback integration and behavioral bias modeling, with implications for AI systems.

Contribution

It compares LLMs and cognitive IBL models in predicting human decisions, highlighting their complementary strengths and proposing integrated approaches for better understanding human behavior.

Findings

LLMs quickly adapt to feedback, improving prediction accuracy.

Cognitive IBL models better capture human exploratory behavior.

IBL models effectively simulate loss aversion bias.

Abstract

Large Language Models (LLMs) have demonstrated their capabilities across various tasks, from language translation to complex reasoning. Understanding and predicting human behavior and biases are crucial for artificial intelligence (AI) assisted systems to provide useful assistance, yet it remains an open question whether these models can achieve this. This paper addresses this gap by leveraging the reasoning and generative capabilities of the LLMs to predict human behavior in two sequential decision-making tasks. These tasks involve balancing between exploitative and exploratory actions and handling delayed feedback, both essential for simulating real-life decision processes. We compare the performance of LLMs with a cognitive instance-based learning (IBL) model, which imitates human experiential decision-making. Our findings indicate that LLMs excel at rapidly incorporating feedback to enhance prediction accuracy. In contrast, the cognitive IBL model better accounts for human exploratory behaviors and effectively captures loss aversion bias, i.e., the tendency to choose a sub-optimal goal with fewer step-cost penalties rather than exploring to find the optimal choice, even with limited experience. The results highlight the benefits of integrating LLMs with cognitive architectures, suggesting that this synergy could enhance the modeling and understanding of complex human decision-making patterns.