Binary Paragraph Vectors

TL;DR

This paper introduces Binary Paragraph Vectors, neural network models that generate compact binary codes for efficient document retrieval, outperforming previous autoencoder-based methods and capturing semantic relevance across domains.

Contribution

The paper proposes Binary Paragraph Vector models that produce short binary codes for documents, improving retrieval speed and accuracy over existing autoencoder-based approaches.

Findings

Binary paragraph vectors outperform autoencoder-based binary codes.

Binary codes effectively capture document semantics across domains.

The combined model enables rapid retrieval of relevant documents.

Abstract

Recently Le & Mikolov described two log-linear models, called Paragraph Vector, that can be used to learn state-of-the-art distributed representations of documents. Inspired by this work, we present Binary Paragraph Vector models: simple neural networks that learn short binary codes for fast information retrieval. We show that binary paragraph vectors outperform autoencoder-based binary codes, despite using fewer bits. We also evaluate their precision in transfer learning settings, where binary codes are inferred for documents unrelated to the training corpus. Results from these experiments indicate that binary paragraph vectors can capture semantics relevant for various domain-specific documents. Finally, we present a model that simultaneously learns short binary codes and longer, real-valued representations. This model can be used to rapidly retrieve a short list of highly relevant documents from a large document collection.