GlitchMiner: Mining Glitch Tokens in Large Language Models via Gradient-based Discrete Optimization

TL;DR

GlitchMiner is a novel gradient-guided framework that effectively detects glitch tokens in large language models by maximizing predictive entropy, outperforming existing methods in accuracy and efficiency.

Contribution

It introduces a behavior-driven, gradient-based optimization approach for glitch token detection that is model-agnostic and scalable.

Findings

Outperforms existing detection methods in accuracy

Demonstrates high query efficiency across multiple LLMs

Provides a generalizable approach for glitch token discovery

Abstract

Glitch tokens, inputs that trigger unpredictable or anomalous behavior in Large Language Models (LLMs), pose significant challenges to model reliability and safety. Existing detection methods primarily rely on heuristic embedding patterns or statistical anomalies within internal representations, limiting their generalizability across different model architectures and potentially missing anomalies that deviate from observed patterns. We introduce GlitchMiner, an behavior-driven framework designed to identify glitch tokens by maximizing predictive entropy. Leveraging a gradient-guided local search strategy, GlitchMiner efficiently explores the discrete token space without relying on model-specific heuristics or large-batch sampling. Extensive experiments across ten LLMs from five major model families demonstrate that GlitchMiner consistently outperforms existing approaches in detection accuracy and query efficiency, providing a generalizable and scalable solution for effective glitch token discovery. Code is available at [https://github.com/wooozihu/GlitchMiner]