A Neurosymbolic Framework for Bias Correction in Convolutional Neural Networks

TL;DR

This paper presents NeSyBiCor, a neurosymbolic framework that corrects biases in CNNs by integrating symbolic concepts and retraining with a semantic similarity loss, enhancing interpretability with minimal accuracy loss.

Contribution

The paper introduces a novel neurosymbolic approach for bias correction in CNNs using ASP constraints and a semantic similarity loss during retraining.

Findings

Successfully corrects biases in CNNs trained on subset classes from Places dataset.

Maintains high accuracy while improving interpretability.

Demonstrates effectiveness of the framework in bias correction.

Abstract

Recent efforts in interpreting Convolutional Neural Networks (CNNs) focus on translating the activation of CNN filters into a stratified Answer Set Program (ASP) rule-sets. The CNN filters are known to capture high-level image concepts, thus the predicates in the rule-set are mapped to the concept that their corresponding filter represents. Hence, the rule-set exemplifies the decision-making process of the CNN w.r.t the concepts that it learns for any image classification task. These rule-sets help understand the biases in CNNs, although correcting the biases remains a challenge. We introduce a neurosymbolic framework called NeSyBiCor for bias correction in a trained CNN. Given symbolic concepts, as ASP constraints, that the CNN is biased towards, we convert the concepts to their corresponding vector representations. Then, the CNN is retrained using our novel semantic similarity loss that pushes the filters away from (or towards) learning the desired/undesired concepts. The final ASP rule-set obtained after retraining, satisfies the constraints to a high degree, thus showing the revision in the knowledge of the CNN. We demonstrate that our NeSyBiCor framework successfully corrects the biases of CNNs trained with subsets of classes from the "Places" dataset while sacrificing minimal accuracy and improving interpretability.