Retrieval-Enhanced Machine Learning

TL;DR

This paper introduces a retrieval-enhanced machine learning framework that leverages information access principles to improve model generalization, scalability, robustness, and interpretability, challenging traditional retrieval conventions.

Contribution

It proposes a generic REML framework that unifies existing models and opens new research directions in integrating information retrieval with machine learning.

Findings

REML improves model robustness and interpretability.

The framework encompasses various existing models as special cases.

Challenges traditional information retrieval assumptions.

Abstract

Although information access systems have long supported people in accomplishing a wide range of tasks, we propose broadening the scope of users of information access systems to include task-driven machines, such as machine learning models. In this way, the core principles of indexing, representation, retrieval, and ranking can be applied and extended to substantially improve model generalization, scalability, robustness, and interpretability. We describe a generic retrieval-enhanced machine learning (REML) framework, which includes a number of existing models as special cases. REML challenges information retrieval conventions, presenting opportunities for novel advances in core areas, including optimization. The REML research agenda lays a foundation for a new style of information access research and paves a path towards advancing machine learning and artificial intelligence.