A Frequency-Based Learning-To-Rank Approach for Personal Digital Traces

TL;DR

This paper introduces a frequency-based learning-to-rank method utilizing LambdaMART and a multidimensional data model to enhance personal digital trace search accuracy, addressing data heterogeneity and lack of training data.

Contribution

It proposes a novel frequency-based learning-to-rank approach using LambdaMART and a multidimensional data model for personal digital traces, with a new training data generation method.

Findings

Improved search accuracy over traditional tools.

Effective handling of heterogeneous digital traces.

Successful application on real user data and email collections.

Abstract

Personal digital traces are constantly produced by connected devices, internet services and interactions. These digital traces are typically small, heterogeneous and stored in various locations in the cloud or on local devices, making it a challenge for users to interact with and search their own data. By adopting a multidimensional data model based on the six natural questions -- what, when, where, who, why and how -- to represent and unify heterogeneous personal digital traces, we can propose a learning-to-rank approach using the state of the art LambdaMART algorithm and frequency-based features that leverage the correlation between content (what), users (who), time (when), location (where) and data source (how) to improve the accuracy of search results. Due to the lack of publicly available personal training data, a combination of known-item query generation techniques and an unsupervised ranking model (field-based BM25) is used to build our own training sets. Experiments performed over a publicly available email collection and a personal digital data trace collection from a real user show that the frequency-based learning approach improves search accuracy when compared with traditional search tools.