Explainability for Vision Foundation Models: A Survey

TL;DR

This survey reviews how foundation models in vision AI challenge and enable explainability, highlighting current research, challenges, evaluation methods, and future directions in integrating XAI with these complex models.

Contribution

It provides a comprehensive categorization and analysis of existing research on explainability in vision foundation models, and discusses future research directions.

Findings

Foundation models are inherently complex and difficult to interpret.

They are increasingly used as tools to build explainable AI systems.

Current evaluation methodologies for explainability are diverse and evolving.

Abstract

As artificial intelligence systems become increasingly integrated into daily life, the field of explainability has gained significant attention. This trend is particularly driven by the complexity of modern AI models and their decision-making processes. The advent of foundation models, characterized by their extensive generalization capabilities and emergent uses, has further complicated this landscape. Foundation models occupy an ambiguous position in the explainability domain: their complexity makes them inherently challenging to interpret, yet they are increasingly leveraged as tools to construct explainable models. In this survey, we explore the intersection of foundation models and eXplainable AI (XAI) in the vision domain. We begin by compiling a comprehensive corpus of papers that bridge these fields. Next, we categorize these works based on their architectural characteristics. We then discuss the challenges faced by current research in integrating XAI within foundation models. Furthermore, we review common evaluation methodologies for these combined approaches. Finally, we present key observations and insights from our survey, offering directions for future research in this rapidly evolving field.