STAR: Detecting Inference-time Backdoors in LLM Reasoning via State-Transition Amplification Ratio

TL;DR

STAR is a novel detection framework that identifies inference-time backdoors in large language models by analyzing probability shifts in reasoning paths, achieving high accuracy and efficiency across multiple models and datasets.

Contribution

This paper introduces STAR, a new method for detecting inference-time backdoors in LLMs by leveraging state-transition amplification ratios, which is robust and more efficient than existing approaches.

Findings

Achieves near-perfect detection AUROC (~1.0) across diverse models and datasets.

Provides approximately 42 times greater efficiency than baseline methods.

Remains robust against adaptive attacks designed to evade detection.

Abstract

Recent LLMs increasingly integrate reasoning mechanisms like Chain-of-Thought (CoT). However, this explicit reasoning exposes a new attack surface for inference-time backdoors, which inject malicious reasoning paths without altering model parameters. Because these attacks generate linguistically coherent paths, they effectively evade conventional detection. To address this, we propose STAR (State-Transition Amplification Ratio), a framework that detects backdoors by analyzing output probability shifts. STAR exploits the statistical discrepancy where a malicious input-induced path exhibits high posterior probability despite a low prior probability in the model's general knowledge. We quantify this state-transition amplification and employ the CUSUM algorithm to detect persistent anomalies. Experiments across diverse models (8B-70B) and five benchmark datasets demonstrate that STAR exhibits robust generalization capabilities, consistently achieving near-perfect performance (AUROC $\approx$ 1.0) with approximately $42\times$ greater efficiency than existing baselines. Furthermore, the framework proves robust against adaptive attacks attempting to bypass detection.