Exploratory Search with Sentence Embeddings

TL;DR

This paper introduces an exploratory search system that uses hierarchical clustering and sentence embeddings to guide users through a document corpus, providing summaries and keyphrases based on semantic similarity.

Contribution

It presents a novel approach combining sentence embeddings with hierarchical clustering for improved exploratory search and document summarization.

Findings

System effectively guides users through large corpora

Summaries and keyphrases are closely aligned with document semantics

Evaluation on personal search history demonstrates practical utility

Abstract

Exploratory search aims to guide users through a corpus rather than pinpointing exact information. We propose an exploratory search system based on hierarchical clusters and document summaries using sentence embeddings. With sentence embeddings, we represent documents as the mean of their embedded sentences, extract summaries containing sentences close to this document representation and extract keyphrases close to the document representation. To evaluate our search system, we scrape our personal search history over the past year and report our experience with the system. We then discuss motivating use cases of an exploratory search system of this nature and conclude with possible directions of future work.