The Road to Generalizable Neuro-Symbolic Learning Should be Paved with Foundation Models

TL;DR

This paper argues that foundation models can enable generalizable neuro-symbolic reasoning, overcoming traditional training challenges and aligning with neuro-symbolic learning's original goals.

Contribution

It proposes leveraging foundation models for neuro-symbolic prompting, highlighting their potential to address limitations of traditional neuro-symbolic methods.

Findings

Foundation models can be combined with symbolic programs for complex reasoning.

Traditional neuro-symbolic learning faces compute, data, and generalization challenges.

Foundation models offer a promising path to achieve neuro-symbolic goals without extensive training.

Abstract

Neuro-symbolic learning was proposed to address challenges with training neural networks for complex reasoning tasks with the added benefits of interpretability, reliability, and efficiency. Neuro-symbolic learning methods traditionally train neural models in conjunction with symbolic programs, but they face significant challenges that limit them to simplistic problems. On the other hand, purely-neural foundation models now reach state-of-the-art performance through prompting rather than training, but they are often unreliable and lack interpretability. Supplementing foundation models with symbolic programs, which we call neuro-symbolic prompting, provides a way to use these models for complex reasoning tasks. Doing so raises the question: What role does specialized model training as part of neuro-symbolic learning have in the age of foundation models? To explore this question, we highlight three pitfalls of traditional neuro-symbolic learning with respect to the compute, data, and programs leading to generalization problems. This position paper argues that foundation models enable generalizable neuro-symbolic solutions, offering a path towards achieving the original goals of neuro-symbolic learning without the downsides of training from scratch.