Artefact Retrieval: Overview of NLP Models with Knowledge Base Access

TL;DR

This paper provides a systematic overview of how NLP models access and incorporate knowledge bases, identifying design patterns, and proposing a unified typology to facilitate transfer across tasks.

Contribution

It introduces a comprehensive typology of artefacts, retrieval mechanisms, and fusion strategies, revealing unexplored combinations and unifying diverse NLP architectures.

Findings

Identifies common design patterns in knowledge base access

Uncovers novel combinations of retrieval and fusion mechanisms

Provides a unified framework for different NLP tasks

Abstract

Many NLP models gain performance by having access to a knowledge base. A lot of research has been devoted to devising and improving the way the knowledge base is accessed and incorporated into the model, resulting in a number of mechanisms and pipelines. Despite the diversity of proposed mechanisms, there are patterns in the designs of such systems. In this paper, we systematically describe the typology of artefacts (items retrieved from a knowledge base), retrieval mechanisms and the way these artefacts are fused into the model. This further allows us to uncover combinations of design decisions that had not yet been tried. Most of the focus is given to language models, though we also show how question answering, fact-checking and knowledgable dialogue models fit into this system as well. Having an abstract model which can describe the architecture of specific models also helps with transferring these architectures between multiple NLP tasks.