Tracing Positional Bias in Financial Decision-Making: Mechanistic Insights from Qwen2.5

TL;DR

This paper introduces a unified framework and benchmark to detect, quantify, and interpret positional bias in financial decision-making by large language models, revealing its prevalence, scale sensitivity, and mechanistic origins.

Contribution

It is the first to systematically analyze and interpret positional bias in open-source Qwen2.5 models within financial contexts, providing actionable insights and a new methodological standard.

Findings

Positional bias is pervasive and scale-sensitive in financial decision models.

Bias resurfaces under nuanced prompts and specific investment scenarios.

Mechanistic interpretability maps bias emergence and propagation within models.

Abstract

The growing adoption of large language models (LLMs) in finance exposes high-stakes decision-making to subtle, underexamined positional biases. The complexity and opacity of modern model architectures compound this risk. We present the first unified framework and benchmark that not only detects and quantifies positional bias in binary financial decisions but also pinpoints its mechanistic origins within open-source Qwen2.5-instruct models (1.5B-14B). Our empirical analysis covers a novel, finance-authentic dataset revealing that positional bias is pervasive, scale-sensitive, and prone to resurfacing under nuanced prompt designs and investment scenarios, with recency and primacy effects revealing new vulnerabilities in risk-laden contexts. Through transparent mechanistic interpretability, we map how and where bias emerges and propagates within the models to deliver actionable, generalizable insights across prompt types and scales. By bridging domain-specific audit with model interpretability, our work provides a new methodological standard for both rigorous bias diagnosis and practical mitigation, establishing essential guidance for responsible and trustworthy deployment of LLMs in financial systems.