LogitDynamics: Reliable ViT Error Detection from Layerwise Logit Trajectories

TL;DR

This paper introduces LogitDynamics, a method using layerwise logit trajectories in Vision Transformers to reliably predict errors with minimal computation, improving confidence estimation in image classification.

Contribution

It presents a simple, effective approach that models class evidence evolution across ViT layers using lightweight heads, enhancing error prediction and generalization.

Findings

Improves AUCPR over baselines across datasets

Shows stronger cross-dataset generalization

Requires minimal additional computation

Abstract

Reliable confidence estimation is critical when deploying vision models. We study error prediction: determining whether an image classifier's output is correct using only signals from a single forward pass. Motivated by internal-signal hallucination detection in large language models, we investigate whether similar depth-wise signals exist in Vision Transformers (ViTs). We propose a simple method that models how class evidence evolves across layers. By attaching lightweight linear heads to intermediate layers, we extract features from the last L layers that capture both the logits of the predicted class and its top-K competitors, as well as statistics describing instability of top-ranked classes across depth. A linear probe trained on these features predicts the error indicator. Across datasets, our method improves or matches AUCPR over baselines and shows stronger cross-dataset generalization while requiring minimal additional computation.