LogicCBMs: Logic-Enhanced Concept-Based Learning

TL;DR

LogicCBMs enhance concept-based neural models by integrating differentiable logic operations, improving expressivity, accuracy, and interpretability over traditional linear concept combinations.

Contribution

We introduce a differentiable logic module for concept bottleneck models, enabling logical composition of concepts for improved expressivity and interpretability.

Findings

Better accuracy on benchmarks and synthetic datasets

Effective intervention capabilities

High interpretability of the model

Abstract

Concept Bottleneck Models (CBMs) provide a basis for semantic abstractions within a neural network architecture. Such models have primarily been seen through the lens of interpretability so far, wherein they offer transparency by inferring predictions as a linear combination of semantic concepts. However, a linear combination is inherently limiting. So we propose the enhancement of concept-based learning models through propositional logic. We introduce a logic module that is carefully designed to connect the learned concepts from CBMs through differentiable logic operations, such that our proposed LogicCBM can go beyond simple weighted combinations of concepts to leverage various logical operations to yield the final predictions, while maintaining end-to-end learnability. Composing concepts using a set of logic operators enables the model to capture inter-concept relations, while simultaneously improving the expressivity of the model in terms of logic operations. Our empirical studies on well-known benchmarks and synthetic datasets demonstrate that these models have better accuracy, perform effective interventions and are highly interpretable.