Minimal neuron ablation triggers catastrophic collapse in the language core of Large Vision-Language Models

TL;DR

This paper uncovers that removing a tiny number of critical neurons in large vision-language models can cause catastrophic failure, highlighting structural vulnerabilities and informing safety considerations.

Contribution

It introduces CAN, a method to identify critical neurons, revealing that very few neurons can induce collapse and that vulnerabilities are mainly in the language component.

Findings

Masking few neurons causes collapse in LVLMs

Critical neurons are mainly in the language model

Vulnerable structures include the down-projection layer

Abstract

Large Vision-Language Models (LVLMs) have shown impressive multimodal understanding capabilities, yet their robustness is poorly understood. In this paper, we investigate the structural vulnerabilities of LVLMs to identify any critical neurons whose removal triggers catastrophic collapse. In this context, we propose CAN, a method to detect Consistently Activated Neurons and to locate critical neurons by progressive masking. Experiments on LLaVA-1.5-7b-hf and InstructBLIP-Vicuna-7b reveal that masking only a tiny portion of the language model's feed-forward networks (just as few as four neurons in extreme cases) suffices to trigger catastrophic collapse. Notably, critical neurons are predominantly localized in the language model rather than in the vision components, and the down-projection layer is a particularly vulnerable structure. We also observe a consistent two-stage collapse pattern: initial expressive degradation followed by sudden, complete collapse. Our findings provide important insights for safety research in LVLMs.