Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification

TL;DR

This paper investigates the interpretability and reusability of CNN kernels in sentence classification, proposing methods to learn semantically coherent kernels and visualize attention for better explanations.

Contribution

It introduces a clustering-based approach to learn semantically coherent kernels and demonstrates kernel reusability across tasks, improving interpretability and efficiency.

Findings

Kernels learned without coherence lack interpretability.

Semantic coherence improves model explainability.

Reusing kernels across tasks maintains competitive performance.

Abstract

The state-of-the-art CNN models give good performance on sentence classification tasks. The purpose of this work is to empirically study desirable properties such as semantic coherence, attention mechanism and reusability of CNNs in these tasks. Semantically coherent kernels are preferable as they are a lot more interpretable for explaining the decision of the learned CNN model. We observe that the learned kernels do not have semantic coherence. Motivated by this observation, we propose to learn kernels with semantic coherence using clustering scheme combined with Word2Vec representation and domain knowledge such as SentiWordNet. We suggest a technique to visualize attention mechanism of CNNs for decision explanation purpose. Reusable property enables kernels learned on one problem to be used in another problem. This helps in efficient learning as only a few additional domain specific filters may have to be learned. We demonstrate the efficacy of our core ideas of learning semantically coherent kernels and leveraging reusable kernels for efficient learning on several benchmark datasets. Experimental results show the usefulness of our approach by achieving performance close to the state-of-the-art methods but with semantic and reusable properties.