# GCAV: A Global Concept Activation Vector Framework for Cross-Layer Consistency in Interpretability

## TL;DR

GCAV introduces a unified framework for consistent cross-layer concept interpretation in neural networks, improving stability and reliability of concept attribution by aligning and integrating concept vectors across layers.

## Contribution

The paper proposes GCAV, a novel method that unifies CAVs across layers using contrastive learning and attention mechanisms, addressing inconsistency issues in interpretability.

## Key findings

- Reduces variance in TCAV scores across layers
- Enhances concept localization accuracy
- Improves robustness against adversarial attacks

## Abstract

Concept Activation Vectors (CAVs) provide a powerful approach for interpreting deep neural networks by quantifying their sensitivity to human-defined concepts. However, when computed independently at different layers, CAVs often exhibit inconsistencies, making cross-layer comparisons unreliable. To address this issue, we propose the Global Concept Activation Vector (GCAV), a novel framework that unifies CAVs into a single, semantically consistent representation. Our method leverages contrastive learning to align concept representations across layers and employs an attention-based fusion mechanism to construct a globally integrated CAV. By doing so, our method significantly reduces the variance in TCAV scores while preserving concept relevance, ensuring more stable and reliable concept attributions. To evaluate the effectiveness of GCAV, we introduce Testing with Global Concept Activation Vectors (TGCAV) as a method to apply TCAV to GCAV-based representations. We conduct extensive experiments on multiple deep neural networks, demonstrating that our method effectively mitigates concept inconsistency across layers, enhances concept localization, and improves robustness against adversarial perturbations. By integrating cross-layer information into a coherent framework, our method offers a more comprehensive and interpretable understanding of how deep learning models encode human-defined concepts. Code and models are available at https://github.com/Zhenghao-He/GCAV.