Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations

TL;DR

This paper explores how cognitive biases influence large language model-driven product recommendations, revealing vulnerabilities that can be exploited to manipulate rankings and highlighting challenges in mitigating such biases.

Contribution

It introduces a novel approach using human psychological principles to analyze and exploit cognitive biases as black-box adversarial strategies in LLM-based recommendations.

Findings

Social proof biases increase recommendation rates and rankings.

Scarcity and exclusivity biases decrease product visibility.

Cognitive biases are deeply embedded in LLMs, causing unpredictable recommendation behavior.

Abstract

The advent of Large Language Models (LLMs) has revolutionized product recommenders, yet their susceptibility to adversarial manipulation poses critical challenges, particularly in real-world commercial applications. Our approach is the first one to tap into human psychological principles, seamlessly modifying product descriptions, making such manipulations hard to detect. In this work, we investigate cognitive biases as black-box adversarial strategies, drawing parallels between their effects on LLMs and human purchasing behavior. Through extensive evaluation across models of varying scale, we find that certain biases, such as social proof, consistently boost product recommendation rate and ranking, while others, like scarcity and exclusivity, surprisingly reduce visibility. Our results demonstrate that cognitive biases are deeply embedded in state-of-the-art LLMs, leading to highly unpredictable behavior in product recommendations and posing significant challenges for effective mitigation.