From Perception to Programs: Regularize, Overparameterize, and Amortize

TL;DR

This paper introduces a neurosymbolic approach that combines perception and program synthesis by developing techniques for end-to-end learning, improving stability and enabling perception of discrete abstractions for symbolic processing.

Contribution

It presents a set of techniques including multitask learning, amortized inference, overparameterization, and program length penalization for joint end-to-end training of perception and program synthesis modules.

Findings

Enhanced stability of gradient-guided program search

Effective joint learning of perception and symbolic processing

Ability to learn perceptual abstractions as discrete symbols

Abstract

Toward combining inductive reasoning with perception abilities, we develop techniques for neurosymbolic program synthesis where perceptual input is first parsed by neural nets into a low-dimensional interpretable representation, which is then processed by a synthesized program. We explore several techniques for relaxing the problem and jointly learning all modules end-to-end with gradient descent: multitask learning; amortized inference; overparameterization; and a differentiable strategy for penalizing lengthy programs. Collectedly this toolbox improves the stability of gradient-guided program search, and suggests ways of learning both how to perceive input as discrete abstractions, and how to symbolically process those abstractions as programs.