Knowledge Base Index Compression via Dimensionality and Precision Reduction

TL;DR

This paper explores methods to compress large knowledge base indexes using dimensionality reduction and precision reduction, achieving significant size reductions with minimal performance loss in question answering tasks.

Contribution

It systematically evaluates PCA, autoencoders, and precision reduction techniques for KB index compression, providing practical guidelines and demonstrating substantial compression ratios.

Findings

PCA is an effective and stable dimensionality reduction method.

Combining PCA with 1-bit precision per dimension is feasible.

Achieved up to 100x compression with 75% of original retrieval performance.

Abstract

Recently neural network based approaches to knowledge-intensive NLP tasks, such as question answering, started to rely heavily on the combination of neural retrievers and readers. Retrieval is typically performed over a large textual knowledge base (KB) which requires significant memory and compute resources, especially when scaled up. On HotpotQA we systematically investigate reducing the size of the KB index by means of dimensionality (sparse random projections, PCA, autoencoders) and numerical precision reduction. Our results show that PCA is an easy solution that requires very little data and is only slightly worse than autoencoders, which are less stable. All methods are sensitive to pre- and post-processing and data should always be centered and normalized both before and after dimension reduction. Finally, we show that it is possible to combine PCA with using 1bit per dimension. Overall we achieve (1) 100$\times$ compression with 75%, and (2) 24$\times$ compression with 92% original retrieval performance.