Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement

TL;DR

This paper introduces a rank-2 subspace method to analyze how external context knowledge and parametric knowledge interact over multiple steps in natural language explanations generated by large language models, revealing nuanced dynamics.

Contribution

It proposes a novel rank-2 subspace approach for disentangling and analyzing multi-step knowledge interactions in LLM-generated explanations, surpassing previous rank-1 models.

Findings

Rank-2 subspace captures diverse knowledge interactions effectively.

Hallucinated generations align strongly with parametric knowledge.

Context-faithful generations maintain balanced PK and CK alignment.

Abstract

Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored. Prior work has largely focused on (1) single-step generation and (2) modelled PK-CK interaction as a binary choice within a rank-1 subspace. This approach overlooks richer interactions and how they unfold over longer generations, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments across four QA datasets and three open-weight LLMs demonstrate that while rank-1 subspaces struggle to represent diverse interactions, our rank-2 formulation captures them effectively, highlighting PK alignment for supportive interactions and CK alignment for conflicting ones. Our multi-step analysis reveals, among others, that hallucinated generations exhibit strong alignment with the PK direction, whereas context-faithful generations maintain a more balanced alignment between PK and CK.