Your AI, Not Your View: The Bias of LLMs in Investment Analysis

TL;DR

This paper investigates the inherent biases of Large Language Models in investment analysis, revealing tendencies towards certain stocks and strategies, and introduces a framework for quantifying and benchmarking these biases.

Contribution

It presents a novel experimental framework for analyzing and measuring biases in LLMs within investment contexts, including a public benchmarking leaderboard.

Findings

Models tend to favor technology stocks, large-cap stocks, and contrarian strategies.

Confirmation bias is prevalent, with models sticking to initial judgments despite counter-evidence.

Distinct, model-specific biases are identified across different LLMs.

Abstract

In finance, Large Language Models (LLMs) face frequent knowledge conflicts arising from discrepancies between their pre-trained parametric knowledge and real-time market data. These conflicts are especially problematic in real-world investment services, where a model's inherent biases can misalign with institutional objectives, leading to unreliable recommendations. Despite this risk, the intrinsic investment biases of LLMs remain underexplored. We propose an experimental framework to investigate emergent behaviors in such conflict scenarios, offering a quantitative analysis of bias in LLM-based investment analysis. Using hypothetical scenarios with balanced and imbalanced arguments, we extract the latent biases of models and measure their persistence. Our analysis, centered on sector, size, and momentum, reveals distinct, model-specific biases. Across most models, a tendency to prefer technology stocks, large-cap stocks, and contrarian strategies is observed. These foundational biases often escalate into confirmation bias, causing models to cling to initial judgments even when faced with increasing counter-evidence. A public leaderboard benchmarking bias across a broader set of models is available at https://linqalpha.com/leaderboard