Multi-step Inference over Unstructured Data

TL;DR

This paper introduces Cora, a neuro-symbolic AI platform that combines LLMs with symbolic reasoning to improve high-stakes decision-making in complex domains, demonstrating superior performance over existing methods.

Contribution

The paper presents a novel neuro-symbolic framework integrating LLMs with symbolic reasoning, specifically applied to high-stakes research tasks, addressing multi-step inference challenges.

Findings

Cora outperforms LLM and RAG baselines in preliminary evaluations.

The system effectively handles complex reasoning and knowledge extraction.

Neuro-symbolic approach enhances logical consistency in decision-making.

Abstract

The advent of Large Language Models (LLMs) and Generative AI has revolutionized natural language applications across various domains. However, high-stakes decision-making tasks in fields such as medical, legal and finance require a level of precision, comprehensiveness, and logical consistency that pure LLM or Retrieval-Augmented-Generation (RAG) approaches often fail to deliver. At Elemental Cognition (EC), we have developed a neuro-symbolic AI platform to tackle these problems. The platform integrates fine-tuned LLMs for knowledge extraction and alignment with a robust symbolic reasoning engine for logical inference, planning and interactive constraint solving. We describe Cora, a Collaborative Research Assistant built on this platform, that is designed to perform complex research and discovery tasks in high-stakes domains. This paper discusses the multi-step inference challenges inherent in such domains, critiques the limitations of existing LLM-based methods, and demonstrates how Cora's neuro-symbolic approach effectively addresses these issues. We provide an overview of the system architecture, key algorithms for knowledge extraction and formal reasoning, and present preliminary evaluation results that highlight Cora's superior performance compared to well-known LLM and RAG baselines.