Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits

TL;DR

This paper presents Reverse Derivative Ascent, a novel categorical method for learning Boolean circuit parameters directly, offering an alternative to neural network approaches, with demonstrated empirical effectiveness on benchmark datasets.

Contribution

It introduces a categorical analogue of gradient methods using reverse differential categories, enabling direct parameter learning of Boolean circuits.

Findings

Effective parameter learning for Boolean circuits demonstrated

Outperforms existing binarised neural network methods

Empirical results on benchmark datasets validate the approach

Abstract

We introduce Reverse Derivative Ascent: a categorical analogue of gradient based methods for machine learning. Our algorithm is defined at the level of so-called reverse differential categories. It can be used to learn the parameters of models which are expressed as morphisms of such categories. Our motivating example is boolean circuits: we show how our algorithm can be applied to such circuits by using the theory of reverse differential categories. Note our methodology allows us to learn the parameters of boolean circuits directly, in contrast to existing binarised neural network approaches. Moreover, we demonstrate its empirical value by giving experimental results on benchmark machine learning datasets.