Object Centric Concept Bottlenecks
David Steinmann, Wolfgang Stammer, Antonia W\"ust, Kristian Kersting

TL;DR
This paper introduces Object-Centric Concept Bottlenecks (OCB), a framework that combines concept-based models with object-centric foundation models to improve interpretability and performance in complex vision tasks.
Contribution
The paper proposes OCB, integrating CBMs with object-centric foundation models, enhancing interpretability and performance in complex visual understanding.
Findings
OCB outperforms traditional CBMs on complex datasets.
Object aggregation strategies significantly impact model performance.
OCB enables interpretable decision-making in complex visual tasks.
Abstract
Developing high-performing, yet interpretable models remains a critical challenge in modern AI. Concept-based models (CBMs) attempt to address this by extracting human-understandable concepts from a global encoding (e.g., image encoding) and then applying a linear classifier on the resulting concept activations, enabling transparent decision-making. However, their reliance on holistic image encodings limits their expressiveness in object-centric real-world settings and thus hinders their ability to solve complex vision tasks beyond single-label classification. To tackle these challenges, we introduce Object-Centric Concept Bottlenecks (OCB), a framework that combines the strengths of CBMs and pre-trained object-centric foundation models, boosting performance and interpretability. We evaluate OCB on complex image datasets and conduct a comprehensive ablation study to analyze key…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
