From Specification to Architecture: A Theory Compiler for Knowledge-Guided Machine Learning

TL;DR

This paper introduces the Theory Compiler, an automated system that translates formal domain theories into neural network architectures with provable consistency, enhancing generalisation and reducing manual effort.

Contribution

It proposes a novel system that automates the translation of domain theories into architectures with formal correctness guarantees, addressing scalability and transferability issues.

Findings

Automated architecture generation from formal theories.

Potential for improved generalisation and data efficiency.

Foundation for scalable, verifiable knowledge-guided machine learning.

Abstract

Theory-guided machine learning has demonstrated that including authentic domain knowledge directly into model design improves performance, sample efficiency and out-of-distribution generalisation. Yet the process by which a formal domain theory is translated into architectural constraints remains entirely manual, specific to each domain formalism, and devoid of any formal correctness guarantee. This translation is non-transferable between domains, not verified, and does not scale. We propose the Theory Compiler: a system that accepts a typed, machine-readable domain theory as input and automatically produces an architecture whose function space is provably constrained to be consistent with that theory by construction, not by regularisation. We identify three foundational open problems whose resolution defines our research agenda: (1) designing a universal theory formalisation language with decidable type-checking; (2) constructing a compositionally correct compilation algorithm from theory primitives to architectural modules; and (3) establishing soundness and completeness criteria for formal verification. We further conjecture that compiled architectures match or exceed manually-designed counterparts in generalisation performance while requiring substantially less training data, a claim we ground in classical statistical learning theory. We argue that recent advances in formal machine learning theory, large language models, and the growth of an interdisciplinary research community have made this paradigm achievable for the first time.