A Comprehensive Survey on the Risks and Limitations of Concept-based Models

TL;DR

This survey reviews the risks, limitations, and recent advances of concept-based models, highlighting challenges like concept leakage and robustness issues, and discusses future research directions for their reliable deployment.

Contribution

It provides a comprehensive overview of the challenges and mitigation strategies for concept-based models, emphasizing their limitations and future research avenues.

Findings

Concept leakage and entangled representations are common issues.

Limited robustness to adversarial perturbations affects reliability.

Recent advances aim to improve model interpretability and robustness.

Abstract

Concept-based Models are a class of inherently explainable networks that improve upon standard Deep Neural Networks by providing a rationale behind their predictions using human-understandable `concepts'. With these models being highly successful in critical applications like medical diagnosis and financial risk prediction, there is a natural push toward their wider adoption in sensitive domains to instill greater trust among diverse stakeholders. However, recent research has uncovered significant limitations in the structure of such networks, their training procedure, underlying assumptions, and their susceptibility to adversarial vulnerabilities. In particular, issues such as concept leakage, entangled representations, and limited robustness to perturbations pose challenges to their reliability and generalization. Additionally, the effectiveness of human interventions in these models remains an open question, raising concerns about their real-world applicability. In this paper, we provide a comprehensive survey on the risks and limitations associated with Concept-based Models. In particular, we focus on aggregating commonly encountered challenges and the architecture choices mitigating these challenges for Supervised and Unsupervised paradigms. We also examine recent advances in improving their reliability and discuss open problems and promising avenues of future research in this domain.