Finding and Editing Multi-Modal Neurons in Pre-Trained Transformers

TL;DR

This paper introduces a new efficient method to identify and edit neurons in multi-modal transformers, improving interpretability and reducing hallucinations by leveraging key neuron properties.

Contribution

It proposes a novel neuron identification technique that avoids costly gradient computations and a knowledge editing approach to mitigate hallucinations in multi-modal LLMs.

Findings

Validated effectiveness through extensive experiments

Identified key properties of multi-modal neurons: sensitivity, specificity, causal-effect

Enhanced interpretability and hallucination mitigation in multi-modal models

Abstract

Understanding the internal mechanisms by which multi-modal large language models (LLMs) interpret different modalities and integrate cross-modal representations is becoming increasingly critical for continuous improvements in both academia and industry. In this paper, we propose a novel method to identify key neurons for interpretability -- how multi-modal LLMs bridge visual and textual concepts for captioning. Our method improves conventional works upon efficiency and applied range by removing needs of costly gradient computation. Based on those identified neurons, we further design a multi-modal knowledge editing method, beneficial to mitigate sensitive words or hallucination. For rationale of our design, we provide theoretical assumption. For empirical evaluation, we have conducted extensive quantitative and qualitative experiments. The results not only validate the effectiveness of our methods, but also offer insightful findings that highlight three key properties of multi-modal neurons: sensitivity, specificity and causal-effect, to shed light for future research.