Automatic Differentiation using Constraint Handling Rules in Prolog

TL;DR

This paper presents a concise Prolog-based implementation of reverse mode automatic differentiation using constraint handling rules, outperforming established frameworks in probabilistic grammar learning tasks.

Contribution

It introduces a novel, minimalistic approach to automatic differentiation in Prolog with significant performance advantages in specific applications.

Findings

Small (50 lines) implementation of reverse mode AD in Prolog

Outperforms Theano and TensorFlow in probabilistic grammar learning

Demonstrates efficiency of CHR-based differentiation in practical tasks

Abstract

Automatic differentiation is a technique which allows a programmer to define a numerical computation via compositions of a broad range of numeric and computational primitives and have the underlying system support the computation of partial derivatives of the result with respect to any of its inputs, without making any finite difference approximations, and without manipulating large symbolic expressions representing the computation. This note describes a novel approach to reverse mode automatic differentiation using constraint logic programmming, specifically, the constraint handling rules (CHR) library of SWI Prolog, resulting in a very small (50 lines of code) implementation. When applied to a differentiation-based implementation of the inside-outside algorithm for parameter learning in probabilistic grammars, the CHR based implementations outperformed two well-known frameworks for optimising differentiable functions, Theano and TensorFlow, by a large margin.