GuideWalk: A Novel Graph-Based Word Embedding for Enhanced Text Classification

TL;DR

GuideWalk introduces a graph-based word embedding method that leverages sentence structure to improve text classification accuracy and robustness, especially with limited training data.

Contribution

The paper proposes the Guided Transition Probability Matrix (GTPM), a novel graph-based embedding approach that captures syntactic, semantic, and hidden information for enhanced text classification.

Findings

GTPM outperforms existing embedding algorithms in classification tasks.

GTPM maintains high performance with only 10% training data.

The method shows an 8% performance decline with limited data, better than baseline methods.

Abstract

One of the prime problems of computer science and machine learning is to extract information efficiently from large-scale, heterogeneous data. Text data, with its syntax, semantics, and even hidden information content, possesses an exceptional place among the data types in concern. The processing of the text data requires embedding, a method of translating the content of the text to numeric vectors. A correct embedding algorithm is the starting point for obtaining the full information content of the text data. In this work, a new text embedding approach, namely the Guided Transition Probability Matrix (GTPM) model is proposed. The model uses the graph structure of sentences to capture different types of information from text data, such as syntactic, semantic, and hidden content. Using random walks on a weighted word graph, GTPM calculates transition probabilities to derive text embedding vectors. The proposed method is tested with real-world data sets and eight well-known and successful embedding algorithms. GTPM shows significantly better classification performance for binary and multi-class datasets than well-known algorithms. Additionally, the proposed method demonstrates superior robustness, maintaining performance with limited (only $10\%$) training data, showing an $8\%$ decline compared to $15-20\%$ for baseline methods.