Faithful-Patchscopes: Understanding and Mitigating Model Bias in Hidden Representations Explanation of Large Language Models

TL;DR

This paper investigates the systematic bias in Large Language Models' explanations generated via Patchscopes, revealing unfaithfulness due to linguistic priors, and proposes BALOR, a logit recalibration method, to improve faithfulness and contextual reliance.

Contribution

The paper introduces a dataset to evaluate Patchscope faithfulness and proposes BALOR, a novel bias mitigation technique that enhances explanation accuracy in LLMs.

Findings

Patchscope faithfulness decreases by 18.84% on biased cases

BALOR improves explanation faithfulness by up to 33%

Bias mitigation enhances reliance on contextual information

Abstract

Large Language Models (LLMs) have demonstrated strong capabilities for hidden representation interpretation through Patchscopes, a framework that uses LLMs themselves to generate human-readable explanations by decoding from internal hidden representations. However, our work shows that LLMs tend to rely on inherent linguistic patterns, which can override contextual information encoded in the hidden representations during decoding. For example, even when a hidden representation encodes the contextual attribute "purple" for "broccoli", LLMs still generate "green" in their explanations, reflecting a strong prior association. This behavior reveals a systematic unfaithfulness in Patchscopes. To systematically study this issue, we first designed a dataset to evaluate the faithfulness of Patchscopes under biased cases, and our results show that there is an 18.84\% faithfulness decrease on average. We then propose Bias Alignment through Logit Recalibration (BALOR), which treats the output logits from an unpatched prompt as capturing model bias and contrasts them with logits obtained under patched contextual information. By recalibrating the logit distribution through this contrast, BALOR suppresses model bias and amplifies contextual information during generation. Experiments across multiple LLMs demonstrate that BALOR consistently outperforms existing baselines, achieving up to 33\% relative performance improvement.