When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models

TL;DR

This paper investigates how routing decisions in Mixture-of-Experts language models can be suboptimal, especially on fragile tokens, and proposes a minimal router update to improve routing utility and task performance.

Contribution

It introduces a counterfactual routing analysis revealing misrouting issues and demonstrates that simple router updates can significantly enhance model performance.

Findings

Standard routing aligns well on confident tokens but poorly on fragile tokens.

Lower-loss alternative routes exist inside the model but are not selected by the current router.

Minimal router-only updates improve task performance on benchmark datasets.

Abstract

Mixture-of-Experts (MoE) language models route each token to a small subset of experts, but whether the routes selected by a trained top-$k$ router are good ones is rarely evaluated directly. Holding the model fixed, we compare each standard route against sampled equal-compute alternatives for the same token and score each by the next-token probability it assigns to the realized token in a verified reasoning trajectory. The result is sharply token-conditional: the standard router is well-aligned with route utility on confident tokens but uninformative on the fragile tokens that drive hard reasoning, where lower-loss equal-compute routes consistently exist inside the frozen model but are not selected. The same pattern holds across Qwen3-30B-A3B, GPT-OSS-20B, DeepSeek-V2-Lite, and OLMoE-1B-7B, and follows structurally from how standard top-$k$ training evaluates routing decisions: the language modeling loss scores only the executed route, and load balancing depends only on aggregate routing statistics. A minimal router-only update to the final-layer router, leaving every expert and every other router frozen, is sufficient to shift pass@K on AIME 2024+2025 and HMMT 2025 for both Qwen3-30B-A3B and GPT-OSS-20B, suggesting that at least part of the failure reflects router-reachable misallocation rather than expert capacity alone.