Dr. Neurosymbolic, or: How I Learned to Stop Worrying and Accept Statistics

TL;DR

This paper aims to bridge the gap between symbolic AI and statistical machine learning by providing a clear, theory-based protocol for designing ML systems with solid guarantees, tailored for experienced symbolic researchers.

Contribution

It offers a concise, theory-oriented guide to understanding when statistical ML can be reliably accepted by the symbolic AI community, emphasizing mathematical modeling over specific algorithms.

Findings

Provides a step-by-step protocol for ML system design with guarantees

Clarifies conditions under which symbolic AI can accept statistical methods

Synthesizes foundational statistical concepts relevant to symbolic researchers

Abstract

The symbolic AI community is increasingly trying to embrace machine learning in neuro-symbolic architectures, yet is still struggling due to cultural barriers. To break the barrier, this rather opinionated personal memo attempts to explain and rectify the conventions in Statistics, Machine Learning, and Deep Learning from the viewpoint of outsiders. It provides a step-by-step protocol for designing a machine learning system that satisfies a minimum theoretical guarantee necessary for being taken seriously by the symbolic AI community, i.e., it discusses "in what condition we can stop worrying and accept statistical machine learning." Unlike most textbooks which are written for students trying to specialize in Stat/ML/DL and willing to accept jargons, this memo is written for experienced symbolic researchers that hear a lot of buzz but are still uncertain and skeptical. Information on Stat/ML/DL is currently too scattered or too noisy to invest in. This memo prioritizes compactness, citations to old papers (many in early 20th century), and concepts that resonate well with symbolic paradigms in order to offer time savings. It prioritizes general mathematical modeling and does not discuss any specific function approximator, such as neural networks (NNs), SVMs, decision trees, etc. Finally, it is open to corrections. Consider this memo as something similar to a blog post taking the form of a paper on Arxiv.