Categories of Differentiable Polynomial Circuits for Machine Learning

TL;DR

This paper introduces polynomial circuits as a new class of differentiable models within reverse derivative categories, providing an axiomatisation and demonstrating their functional completeness for machine learning tasks.

Contribution

It presents a formal framework for polynomial circuits in RDCs, including axioms and completeness proofs, and explores their application to discrete-value machine learning.

Findings

Polynomial circuits can be axiomatised within RDCs.

The paper proves the functional completeness of polynomial circuits.

Application to discrete-value machine learning is discussed.

Abstract

Reverse derivative categories (RDCs) have recently been shown to be a suitable semantic framework for studying machine learning algorithms. Whereas emphasis has been put on training methodologies, less attention has been devoted to particular \emph{model classes}: the concrete categories whose morphisms represent machine learning models. In this paper we study presentations by generators and equations of classes of RDCs. In particular, we propose \emph{polynomial circuits} as a suitable machine learning model. We give an axiomatisation for these circuits and prove a functional completeness result. Finally, we discuss the use of polynomial circuits over specific semirings to perform machine learning with discrete values.