A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models

TL;DR

This study investigates the faithfulness of chain-of-thought reasoning in large vision-language models, revealing biases, inconsistencies, and challenges in articulating implicit cues, with a novel evaluation framework for detailed analysis.

Contribution

Introduces a fine-grained evaluation pipeline for analyzing bias articulation and faithfulness in vision-language models' reasoning processes, uncovering new phenomena and insights.

Findings

Image-based biases are rarely articulated compared to text-based biases.

Many models show 'inconsistent' reasoning, switching answers abruptly.

Current models struggle with implicit cues in reasoning.

Abstract

Chain-of-thought (CoT) reasoning enhances performance of large language models, but questions remain about whether these reasoning traces faithfully reflect the internal processes of the model. We present the first comprehensive study of CoT faithfulness in large vision-language models (LVLMs), investigating how both text-based and previously unexplored image-based biases affect reasoning and bias articulation. Our work introduces a novel, fine-grained evaluation pipeline for categorizing bias articulation patterns, enabling significantly more precise analysis of CoT reasoning than previous methods. This framework reveals critical distinctions in how models process and respond to different types of biases, providing new insights into LVLM CoT faithfulness. Our findings reveal that subtle image-based biases are rarely articulated compared to explicit text-based ones, even in models specialized for reasoning. Additionally, many models exhibit a previously unidentified phenomenon we term ``inconsistent'' reasoning - correctly reasoning before abruptly changing answers, serving as a potential canary for detecting biased reasoning from unfaithful CoTs. We then apply the same evaluation pipeline to revisit CoT faithfulness in LLMs across various levels of implicit cues. Our findings reveal that current language-only reasoning models continue to struggle with articulating cues that are not overtly stated.