Lbl2Vec: An Embedding-Based Approach for Unsupervised Document Retrieval on Predefined Topics

TL;DR

Lbl2Vec introduces an unsupervised embedding-based method for retrieving and classifying documents by predefined topics using only keywords, achieving high accuracy without labeled data or extensive preprocessing.

Contribution

The paper presents a novel approach that jointly learns document and word embeddings solely from unlabeled data for effective topic-based document retrieval and classification.

Findings

Achieved average AUC of 0.95 and 0.92 on two datasets.

Improved F1 scores from 76.6 to 82.7 and 61.0 to 75.1 over baselines.

Requires minimal text preprocessing and no labeled data.

Abstract

In this paper, we consider the task of retrieving documents with predefined topics from an unlabeled document dataset using an unsupervised approach. The proposed unsupervised approach requires only a small number of keywords describing the respective topics and no labeled document. Existing approaches either heavily relied on a large amount of additionally encoded world knowledge or on term-document frequencies. Contrariwise, we introduce a method that learns jointly embedded document and word vectors solely from the unlabeled document dataset in order to find documents that are semantically similar to the topics described by the keywords. The proposed method requires almost no text preprocessing but is simultaneously effective at retrieving relevant documents with high probability. When successively retrieving documents on different predefined topics from publicly available and commonly used datasets, we achieved an average area under the receiver operating characteristic curve value of 0.95 on one dataset and 0.92 on another. Further, our method can be used for multiclass document classification, without the need to assign labels to the dataset in advance. Compared with an unsupervised classification baseline, we increased F1 scores from 76.6 to 82.7 and from 61.0 to 75.1 on the respective datasets. For easy replication of our approach, we make the developed Lbl2Vec code publicly available as a ready-to-use tool under the 3-Clause BSD license.