A database approach to information retrieval: The remarkable relationship between language models and region models

TL;DR

This paper unifies region models and language models in information retrieval, revealing a remarkable one-to-one relationship that enhances understanding and application of search technologies across various domains.

Contribution

It introduces a unified model linking region and language models, enabling complex language-based queries within structured document retrieval.

Findings

A one-to-one correspondence between region queries and language models

Unified model applies to ad-hoc, cross-language, video, and web search

Simplifies development of complex language retrieval applications

Abstract

In this report, we unify two quite distinct approaches to information retrieval: region models and language models. Region models were developed for structured document retrieval. They provide a well-defined behaviour as well as a simple query language that allows application developers to rapidly develop applications. Language models are particularly useful to reason about the ranking of search results, and for developing new ranking approaches. The unified model allows application developers to define complex language modeling approaches as logical queries on a textual database. We show a remarkable one-to-one relationship between region queries and the language models they represent for a wide variety of applications: simple ad-hoc search, cross-language retrieval, video retrieval, and web search.