Unsupervised decoding of encoded reasoning using language model interpretability

TL;DR

This paper demonstrates that existing interpretability techniques, especially logit lens analysis combined with paraphrasing, can effectively decode hidden reasoning processes in large language models, even when reasoning is encoded.

Contribution

It introduces a controlled testbed for evaluating interpretability methods on encoded reasoning and develops an unsupervised decoding pipeline that reconstructs reasoning from internal activations.

Findings

Logit lens analysis effectively decodes encoded reasoning.

Intermediate-to-late layers contain most decoding information.

Unsupervised pipeline achieves high accuracy in reconstructing reasoning.

Abstract

As large language models become increasingly capable, there is growing concern that they may develop reasoning processes that are encoded or hidden from human oversight. To investigate whether current interpretability techniques can penetrate such encoded reasoning, we construct a controlled testbed by fine-tuning a reasoning model (DeepSeek-R1-Distill-Llama-70B) to perform chain-of-thought reasoning in ROT-13 encryption while maintaining intelligible English outputs. We evaluate mechanistic interpretability methods--in particular, logit lens analysis--on their ability to decode the model's hidden reasoning process using only internal activations. We show that logit lens can effectively translate encoded reasoning, with accuracy peaking in intermediate-to-late layers. Finally, we develop a fully unsupervised decoding pipeline that combines logit lens with automated paraphrasing, achieving substantial accuracy in reconstructing complete reasoning transcripts from internal model representations. These findings suggest that current mechanistic interpretability techniques may be more robust to simple forms of encoded reasoning than previously understood. Our work provides an initial framework for evaluating interpretability methods against models that reason in non-human-readable formats, contributing to the broader challenge of maintaining oversight over increasingly capable AI systems.