SAUCE: Truncated Sparse Document Signature Bit-Vectors for Fast Web-Scale Corpus Expansion

TL;DR

SAUCE introduces a novel sparse document signature method that enables fast, scalable web-scale corpus expansion from limited seed data, effectively capturing domain-specific terms with reduced computational costs.

Contribution

The paper proposes SAUCE, a truncated sparse bit-vector representation for efficient corpus expansion, addressing computational challenges and long-tail term coverage in domain-specific text retrieval.

Findings

SAUCE significantly reduces computational time compared to traditional methods.

SAUCE achieves high lexical coverage of domain-specific terms.

Experimental results validate SAUCE's effectiveness in large-scale corpus expansion.

Abstract

Recent advances in text representation have shown that training on large amounts of text is crucial for natural language understanding. However, models trained without predefined notions of topical interest typically require careful fine-tuning when transferred to specialized domains. When a sufficient amount of within-domain text may not be available, expanding a seed corpus of relevant documents from large-scale web data poses several challenges. First, corpus expansion requires scoring and ranking each document in the collection, an operation that can quickly become computationally expensive as the web corpora size grows. Relying on dense vector spaces and pairwise similarity adds to the computational expense. Secondly, as the domain concept becomes more nuanced, capturing the long tail of domain-specific rare terms becomes non-trivial, especially under limited seed corpora scenarios. In this paper, we consider the problem of fast approximate corpus expansion given a small seed corpus with a few relevant documents as a query, with the goal of capturing the long tail of a domain-specific set of concept terms. To efficiently collect large-scale domain-specific corpora with limited relevance feedback, we propose a novel truncated sparse document bit-vector representation, termed Signature Assisted Unsupervised Corpus Expansion (SAUCE). Experimental results show that SAUCE can reduce the computational burden while ensuring high within-domain lexical coverage.