FOCUS: Fused Observation of Channels for Unveiling Spectra

TL;DR

FOCUS is a novel framework that enhances the interpretability of Vision Transformers in hyperspectral imaging by generating stable, spectral-aware saliency maps efficiently without modifying the model.

Contribution

It introduces class-specific spectral prompts and a learnable [SINK] token to improve spectral interpretability and stability in frozen ViTs for hyperspectral data.

Findings

Increases band-level IoU by 15%

Reduces attention collapse by over 40%

Produces saliency maps aligning with expert annotations

Abstract

Hyperspectral imaging (HSI) captures hundreds of narrow, contiguous wavelength bands, making it a powerful tool in biology, agriculture, and environmental monitoring. However, interpreting Vision Transformers (ViTs) in this setting remains largely unexplored due to two key challenges: (1) existing saliency methods struggle to capture meaningful spectral cues, often collapsing attention onto the class token, and (2) full-spectrum ViTs are computationally prohibitive for interpretability, given the high-dimensional nature of HSI data. We present FOCUS, the first framework that enables reliable and efficient spatial-spectral interpretability for frozen ViTs. FOCUS introduces two core components: class-specific spectral prompts that guide attention toward semantically meaningful wavelength groups, and a learnable [SINK] token trained with an attraction loss to absorb noisy or redundant attention. Together, these designs make it possible to generate stable and interpretable 3D saliency maps and spectral importance curves in a single forward pass, without any gradient backpropagation or backbone modification. FOCUS improves band-level IoU by 15 percent, reduces attention collapse by over 40 percent, and produces saliency results that align closely with expert annotations. With less than 1 percent parameter overhead, our method makes high-resolution ViT interpretability practical for real-world hyperspectral applications, bridging a long-standing gap between black-box modeling and trustworthy HSI decision-making.