Geometry of Program Synthesis

TL;DR

This paper introduces a novel geometric perspective on program synthesis, viewing programs as singularities in analytic varieties, and explores implications for neural networks, phase transitions, and generalisation, supported by initial experiments.

Contribution

It presents a new geometric framework for understanding program synthesis, connecting it to concepts like phase transitions and complexity, and provides initial empirical evidence.

Findings

Programs as singularities of analytic varieties

Neural networks as a subset within this framework

Initial experiments supporting the new perspective

Abstract

We re-evaluate universal computation based on the synthesis of Turing machines. This leads to a view of programs as singularities of analytic varieties or, equivalently, as phases of the Bayesian posterior of a synthesis problem. This new point of view reveals unexplored directions of research in program synthesis, of which neural networks are a subset, for example in relation to phase transitions, complexity and generalisation. We also lay the empirical foundations for these new directions by reporting on our implementation in code of some simple experiments.