A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledge

TL;DR

This paper investigates the causes of negative bias in large language models, revealing that prompt format and knowledge sufficiency significantly influence biased responses, and explores methods to mitigate this bias.

Contribution

It introduces a systematic evaluation pipeline based on parametric knowledge and analyzes how prompt design affects negative bias in LLMs, offering new insights for bias mitigation.

Findings

Negative bias is influenced more by prompt format than semantics.

Lack of sufficient knowledge leads to increased negative responses.

Providing relevant context and 'I don't know' options reduces negative bias.

Abstract

Negative bias refers to the tendency of large language models (LLMs) to excessively generate negative responses in binary decision tasks (e.g., yes-no question answering). Previous research has focused on detecting and addressing negative attention heads that induce negative bias. However, the underlying detailed factors influencing negative bias remain underexplored. In this paper, we demonstrate that LLMs exhibit format-level negative bias, meaning the prompt format more influences their responses than the semantics of the negative response. For the fine-grained study of the negative bias, we introduce a pipeline for constructing the evaluation set, which systematically categorizes the dataset into three subsets based on the model's parametric knowledge: correct, incorrect, and insufficient relevant knowledge. Through analysis of this evaluation set, we identify a shortcut behavior in which models tend to generate negative responses when they lack sufficient knowledge to answer a yes-no question, leading to negative bias. We further examine how negative bias changes under various prompting scenarios related to parametric knowledge. We observe that providing relevant context and offering an "I don't know" option generally reduces negative bias, whereas chain-of-thought prompting tends to amplify the bias. Finally, we demonstrate that the degree of negative bias can vary depending on the type of prompt, which influences the direction of the response. Our work reveals the various factors that influence negative bias, providing critical insights for mitigating it in LLMs.