MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning

TL;DR

MRKL systems integrate large language models with external knowledge sources and discrete reasoning modules to overcome limitations of pure neural models, enabling more robust AI reasoning and knowledge tasks.

Contribution

This paper introduces the MRKL architecture, a modular neuro-symbolic system combining neural models with symbolic reasoning and knowledge modules, advancing AI capabilities.

Findings

Demonstrates the feasibility of integrating neural and symbolic components

Addresses key limitations of large language models in reasoning tasks

Presents an implementation called Jurassic-X by AI21 Labs

Abstract

Huge language models (LMs) have ushered in a new era for AI, serving as a gateway to natural-language-based knowledge tasks. Although an essential element of modern AI, LMs are also inherently limited in a number of ways. We discuss these limitations and how they can be avoided by adopting a systems approach. Conceptualizing the challenge as one that involves knowledge and reasoning in addition to linguistic processing, we define a flexible architecture with multiple neural models, complemented by discrete knowledge and reasoning modules. We describe this neuro-symbolic architecture, dubbed the Modular Reasoning, Knowledge and Language (MRKL, pronounced "miracle") system, some of the technical challenges in implementing it, and Jurassic-X, AI21 Labs' MRKL system implementation.