Improve Document Embedding for Text Categorization Through Deep Siamese Neural Network

TL;DR

This paper introduces a deep Siamese neural network approach to generate low-dimensional, semantically meaningful document embeddings that improve text categorization performance, demonstrated on the BBC news dataset.

Contribution

It proposes a novel deep Siamese neural network method for learning document representations that enhance text classification accuracy.

Findings

Outperforms conventional text embedding methods

Achieves higher accuracy on BBC news dataset

Effective in capturing semantic relevance in document embeddings

Abstract

Due to the increasing amount of data on the internet, finding a highly-informative, low-dimensional representation for text is one of the main challenges for efficient natural language processing tasks including text classification. This representation should capture the semantic information of the text while retaining their relevance level for document classification. This approach maps the documents with similar topics to a similar space in vector space representation. To obtain representation for large text, we propose the utilization of deep Siamese neural networks. To embed document relevance in topics in the distributed representation, we use a Siamese neural network to jointly learn document representations. Our Siamese network consists of two sub-network of multi-layer perceptron. We examine our representation for the text categorization task on BBC news dataset. The results show that the proposed representations outperform the conventional and state-of-the-art representations in the text classification task on this dataset.