HOUDINI: Lifelong Learning as Program Synthesis

TL;DR

HOUDINI introduces a neurosymbolic lifelong learning framework that combines program synthesis with neural networks to improve transfer learning and task generalization in algorithmic and perception tasks.

Contribution

It proposes a novel program synthesis approach that integrates gradient descent with combinatorial search for lifelong learning of complex tasks.

Findings

HOUDINI outperforms traditional transfer learning methods.

Typed representations significantly speed up the search process.

Effective transfer of high-level concepts across tasks.

Abstract

We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing high-level concepts across domains and learning complex procedures are key challenges in lifelong learning. We show that a program synthesis approach that combines gradient descent with combinatorial search over programs can be a more effective response to these challenges than purely neural methods. Our framework, called HOUDINI, represents neural networks as strongly typed, differentiable functional programs that use symbolic higher-order combinators to compose a library of neural functions. Our learning algorithm consists of: (1) a symbolic program synthesizer that performs a type-directed search over parameterized programs, and decides on the library functions to reuse, and the architectures to combine them, while learning a sequence of tasks; and (2) a neural module that trains these programs using stochastic gradient descent. We evaluate HOUDINI on three benchmarks that combine perception with the algorithmic tasks of counting, summing, and shortest-path computation. Our experiments show that HOUDINI transfers high-level concepts more effectively than traditional transfer learning and progressive neural networks, and that the typed representation of networks significantly accelerates the search.