The Geometries of Truth Are Orthogonal Across Tasks

TL;DR

This paper investigates the geometry of correct answers in LLM activations, revealing that task-specific geometries do not transfer across different tasks, challenging previous assumptions about universal truth representations.

Contribution

It demonstrates that geometries of truth are task-dependent and do not generalize across tasks, highlighting limitations of current linear probing methods.

Findings

Linear classifiers trained on different tasks have little overlap.

Probes with sparsity regularization have nearly disjoint supports.

Activation clusters are task-specific and separated across tasks.

Abstract

Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on their reliability. Recent works have proposed examining the activations produced by an LLM at inference time to assess whether its answer to a question is correct. Some works claim that a "geometry of truth" can be learned from examples, in the sense that the activations that generate correct answers can be distinguished from those leading to mistakes with a linear classifier. In this work, we underline a limitation of these approaches: we observe that these "geometries of truth" are intrinsically task-dependent and fail to transfer across tasks. More precisely, we show that linear classifiers trained across distinct tasks share little similarity and, when trained with sparsity-enforcing regularizers, have almost disjoint supports. We show that more sophisticated approaches (e.g., using mixtures of probes and tasks) fail to overcome this limitation, likely because activation vectors commonly used to classify answers form clearly separated clusters when examined across tasks.