ReLope: KL-Regularized LoRA Probes for Multimodal LLM Routing

TL;DR

This paper introduces ReLope, a KL-regularized LoRA probe, and an attention-based approach to improve routing accuracy in multimodal large language models by enhancing hidden state signals.

Contribution

The paper proposes novel probe designs, including Attention Probe and KL-Regularized LoRA Probe, to improve correctness signal extraction in multimodal LLM routing.

Findings

ReLope outperforms baseline probes in multimodal LLM routing tasks.

Attention aggregation improves correctness signal separability.

KL regularization enhances routing-aware representation learning.

Abstract

Routing has emerged as a promising strategy for balancing performance and cost in large language model (LLM) systems that combine lightweight models with powerful but expensive large models. Recent studies show that \emph{probe routing}, which predicts the correctness of a small model using its hidden states, provides an effective solution in text-only LLMs. However, we observe that these probes degrade substantially when applied to multimodal LLMs (MLLMs). Through empirical analysis, we find that the presence of visual inputs weakens the separability of correctness signals in hidden states, making them harder to extract using standard probe designs. To address this challenge, we introduce two complementary approaches for improving probe routing in MLLMs. First, we propose the \emph{Attention Probe}, which aggregates hidden states from the preceding layer based on attention scores to recover distributed correctness signals. Second, we present the \emph{KL-Regularized LoRA Probe (ReLope)}, which inserts a lightweight LoRA adapter and applies a KL regularizer to learn routing-aware representations. Comprehensive experiments show that our methods consistently outperform baselines, suggesting that improving the quality of hidden states is key to effective routing in MLLMs. Our code is available at https://github.com/Spinozaaa/ReLope.