Achieving Data Efficient Neural Networks with Hybrid Concept-based Models

TL;DR

This paper introduces hybrid concept-based neural network models that leverage additional concept information for more data-efficient training, demonstrating improved accuracy and robustness in sparse data scenarios.

Contribution

The paper proposes novel hybrid concept-based models and an algorithm for adversarial concept attacks, advancing interpretability and robustness in concept-based learning.

Findings

Hybrid models outperform standard models in accuracy, especially with sparse data.

Concept-based models are vulnerable to adversarial attacks that preserve concepts but alter class predictions.

New datasets with concept labels facilitate evaluation of concept-based models.

Abstract

Most datasets used for supervised machine learning consist of a single label per data point. However, in cases where more information than just the class label is available, would it be possible to train models more efficiently? We introduce two novel model architectures, which we call hybrid concept-based models, that train using both class labels and additional information in the dataset referred to as concepts. In order to thoroughly assess their performance, we introduce ConceptShapes, an open and flexible class of datasets with concept labels. We show that the hybrid concept-based models outperform standard computer vision models and previously proposed concept-based models with respect to accuracy, especially in sparse data settings. We also introduce an algorithm for performing adversarial concept attacks, where an image is perturbed in a way that does not change a concept-based model's concept predictions, but changes the class prediction. The existence of such adversarial examples raises questions about the interpretable qualities promised by concept-based models.