Enhancing Pre-trained Representation Classifiability can Boost its Interpretability

TL;DR

This paper introduces a new metric called Inherent Interpretability Score (IIS) to evaluate how well pre-trained visual representations balance interpretability and classifiability, revealing a positive correlation between the two.

Contribution

It proposes IIS to quantify representation interpretability and demonstrates that enhancing classifiability also improves interpretability in pre-trained vision models.

Findings

Higher classifiability correlates with increased interpretability.

Fine-tuning with interpretability maximization boosts classifiability.

Interpretability-based predictions cause less accuracy degradation.

Abstract

The visual representation of a pre-trained model prioritizes the classifiability on downstream tasks, while the widespread applications for pre-trained visual models have posed new requirements for representation interpretability. However, it remains unclear whether the pre-trained representations can achieve high interpretability and classifiability simultaneously. To answer this question, we quantify the representation interpretability by leveraging its correlation with the ratio of interpretable semantics within the representations. Given the pre-trained representations, only the interpretable semantics can be captured by interpretations, whereas the uninterpretable part leads to information loss. Based on this fact, we propose the Inherent Interpretability Score (IIS) that evaluates the information loss, measures the ratio of interpretable semantics, and quantifies the representation interpretability. In the evaluation of the representation interpretability with different classifiability, we surprisingly discover that the interpretability and classifiability are positively correlated, i.e., representations with higher classifiability provide more interpretable semantics that can be captured in the interpretations. This observation further supports two benefits to the pre-trained representations. First, the classifiability of representations can be further improved by fine-tuning with interpretability maximization. Second, with the classifiability improvement for the representations, we obtain predictions based on their interpretations with less accuracy degradation. The discovered positive correlation and corresponding applications show that practitioners can unify the improvements in interpretability and classifiability for pre-trained vision models. Codes are available at https://github.com/ssfgunner/IIS.