Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

TL;DR

This study evaluates whether mechanistic interpretability methods can effectively correct language model errors, finding that despite encoding task knowledge, these methods fail to reliably translate internal representations into accurate outputs.

Contribution

The paper systematically tests four mechanistic interpretability methods for error correction, revealing their limitations in translating internal knowledge into improved model outputs.

Findings

Linear probes achieve 98.2% AUROC but only 45.1% output sensitivity.

Concept bottleneck steering corrects 20% of hazards but disrupts 53% of correct detections.

Current interpretability methods cannot reliably convert internal knowledge into correct outputs.

Abstract

Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.