Feature-level Interaction Explanations in Multimodal Transformers

TL;DR

This paper introduces FL-I2MoE, a structured explanation method for multimodal Transformers that identifies and quantifies feature interactions, such as synergy and redundancy, to improve interpretability and faithfulness.

Contribution

We propose FL-I2MoE, a novel Mixture-of-Experts layer that explicitly separates and explains feature interactions in multimodal Transformers, enhancing interpretability.

Findings

FL-I2MoE produces more interaction-specific importance patterns.

Removing high-scoring pairs degrades model performance more than random removal.

The method effectively quantifies synergistic and redundant feature pairs.

Abstract

Multimodal Transformers often produce predictions without clarifying how different modalities jointly support a decision. Most existing multimodal explainable AI (MXAI) methods extend unimodal saliency to multimodal backbones, highlighting important tokens or patches within each modality, but they rarely pinpoint which cross-modal feature pairs provide complementary evidence (synergy) or serve as reliable backups (redundancy). We present Feature-level I2MoE (FL-I2MoE), a structured Mixture-of-Experts layer that operates directly on token/patch sequences from frozen pretrained encoders and explicitly separates unique, synergistic, and redundant evidence at the feature level. We further develop an expert-wise explanation pipeline that combines attribution with top-K% masking to assess faithfulness, and we introduce Monte Carlo interaction probes to quantify pairwise behavior: the Shapley Interaction Index (SII) to score synergistic pairs and a redundancy-gap score to capture substitutable (redundant) pairs. Across three benchmarks (MMIMDb, ENRICO, and MMHS150K), FL-I2MoE yields more interactionspecific and concentrated importance patterns than a dense Transformer with the same encoders. Finally, pair-level masking shows that removing pairs ranked by SII or redundancy-gap degrades performance more than masking randomly chosen pairs under the same budget, supporting that the identified interactions are causally relevant.