ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

TL;DR

This paper introduces a large, anonymized click dataset linked to the TREC Deep Learning corpus, enabling new research opportunities in search ranking and query analysis.

Contribution

It releases a sizable, privacy-preserving click dataset aligned with TREC DL, facilitating academic research on search behavior and ranking models.

Findings

Dataset contains 1.4 million URLs and 18 million query connections.

Augmentation with click data significantly increases training data size.

Preliminary experiments show improved ranking performance with the dataset.

Abstract

Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been publicly released for academic use, because they can be too revealing of personally or commercially sensitive information. This paper describes a click data release related to the TREC Deep Learning Track document corpus. After aggregation and filtering, including a k-anonymity requirement, we find 1.4 million of the TREC DL URLs have 18 million connections to 10 million distinct queries. Our dataset of these queries and connections to TREC documents is of similar size to proprietary datasets used in previous papers on query mining and ranking. We perform some preliminary experiments using the click data to augment the TREC DL training data, offering by comparison: 28x more queries, with 49x more connections to 4.4x more URLs in the corpus. We present a description of the dataset's generation process, characteristics, use in ranking and suggest other potential uses.