Sparks of Explainability: Recent Advancements in Explaining Large Vision Models

TL;DR

This paper reviews recent advancements in explaining large vision models, proposing new methods for attribution, concept extraction, and aligning models with human reasoning to enhance interpretability.

Contribution

It introduces novel attribution metrics, the EVA formal guarantees, and the CRAFT and MACO methods for concept extraction and visualization, advancing explainability techniques.

Findings

Attribution methods struggle to clarify 'what' models perceive in complex scenarios.

Sobol indices and stability metrics reduce computation time for attribution.

Concept-based explanations improve understanding of model decisions.

Abstract

This thesis explores advanced approaches to improve explainability in computer vision by analyzing and modeling the features exploited by deep neural networks. Initially, it evaluates attribution methods, notably saliency maps, by introducing a metric based on algorithmic stability and an approach utilizing Sobol indices, which, through quasi-Monte Carlo sequences, allows a significant reduction in computation time. In addition, the EVA method offers a first formulation of attribution with formal guarantees via verified perturbation analysis. Experimental results indicate that in complex scenarios these methods do not provide sufficient understanding, particularly because they identify only "where" the model focuses without clarifying "what" it perceives. Two hypotheses are therefore examined: aligning models with human reasoning -- through the introduction of a training routine that integrates the imitation of human explanations and optimization within the space of 1-Lipschitz functions -- and adopting a conceptual explainability approach. The CRAFT method is proposed to automate the extraction of the concepts used by the model and to assess their importance, complemented by MACO, which enables their visualization. These works converge towards a unified framework, illustrated by an interactive demonstration applied to the 1000 ImageNet classes in a ResNet model.