SECNN: Squeeze-and-Excitation Convolutional Neural Network for Sentence Classification

TL;DR

SECNN enhances sentence classification by integrating squeeze-and-excitation attention with CNNs, allowing the model to learn the importance of different feature maps and improve performance on NLP tasks.

Contribution

This work introduces SECNN, a novel model combining CNNs with channel attention mechanisms for improved sentence classification accuracy.

Findings

SECNN outperforms traditional CNN models on benchmark datasets.

Channel attention improves the model's ability to focus on important features.

SECNN achieves state-of-the-art results in sentence classification tasks.

Abstract

Sentence classification is one of the basic tasks of natural language processing. Convolution neural network (CNN) has the ability to extract n-grams features through convolutional filters and capture local correlations between consecutive words in parallel, so CNN is a popular neural network architecture to dealing with the task. But restricted by the width of convolutional filters, it is difficult for CNN to capture long term contextual dependencies. Attention is a mechanism that considers global information and pays more attention to keywords in sentences, thus attention mechanism is cooperated with CNN network to improve performance in sentence classification task. In our work, we don't focus on keyword in a sentence, but on which CNN's output feature map is more important. We propose a Squeeze-and-Excitation Convolutional neural Network (SECNN) for sentence classification. SECNN takes the feature maps from multiple CNN as different channels of sentence representation, and then, we can utilize channel attention mechanism, that is SE attention mechanism, to enable the model to learn the attention weights of different channel features. The results show that our model achieves advanced performance in the sentence classification task.