A Token-wise CNN-based Method for Sentence Compression

TL;DR

This paper introduces a token-wise CNN model with BERT features for sentence compression, offering a faster alternative to RNN-based methods while maintaining comparable performance.

Contribution

The paper presents a novel CNN-based approach for sentence compression that significantly improves processing speed over traditional RNN models, incorporating BERT features for better accuracy.

Findings

CNN-based model is ten times faster than RNN-based models.

The model achieves comparable compression quality to RNN approaches.

Incorporating BERT features enhances sentence compression performance.

Abstract

Sentence compression is a Natural Language Processing (NLP) task aimed at shortening original sentences and preserving their key information. Its applications can benefit many fields e.g. one can build tools for language education. However, current methods are largely based on Recurrent Neural Network (RNN) models which suffer from poor processing speed. To address this issue, in this paper, we propose a token-wise Convolutional Neural Network, a CNN-based model along with pre-trained Bidirectional Encoder Representations from Transformers (BERT) features for deletion-based sentence compression. We also compare our model with RNN-based models and fine-tuned BERT. Although one of the RNN-based models outperforms marginally other models given the same input, our CNN-based model was ten times faster than the RNN-based approach.