Exploring the generalization of LLM truth directions on conversational formats

TL;DR

This paper investigates whether the concept of a universal truth direction in LLMs generalizes across different conversational formats, revealing limitations and proposing a simple fix to improve lie detection consistency.

Contribution

The study demonstrates the limits of truth direction generalization in longer conversations and introduces a key phrase method to enhance lie detection across formats.

Findings

Good generalization in short conversations ending with a lie

Poor generalization in longer conversations with early lies

Adding a fixed key phrase improves lie detection generalization

Abstract

Several recent works argue that LLMs have a universal truth direction where true and false statements are linearly separable in the activation space of the model. It has been demonstrated that linear probes trained on a single hidden state of the model already generalize across a range of topics and might even be used for lie detection in LLM conversations. In this work we explore how this truth direction generalizes between various conversational formats. We find good generalization between short conversations that end on a lie, but poor generalization to longer formats where the lie appears earlier in the input prompt. We propose a solution that significantly improves this type of generalization by adding a fixed key phrase at the end of each conversation. Our results highlight the challenges towards reliable LLM lie detectors that generalize to new settings.