Computing with Categories in Machine Learning

TL;DR

This paper introduces DisCoPyro, a categorical structure learning framework that integrates category theory with variational inference, aiming to enhance program learning in machine learning and potentially advance artificial general intelligence.

Contribution

It presents a novel framework combining categorical structures with amortized variational inference, bridging abstract mathematics and practical machine learning applications.

Findings

DisCoPyro demonstrates competitive performance in program learning tasks.

The framework effectively integrates categorical structures with inference methods.

Potential to contribute to artificial general intelligence development.

Abstract

Category theory has been successfully applied in various domains of science, shedding light on universal principles unifying diverse phenomena and thereby enabling knowledge transfer between them. Applications to machine learning have been pursued recently, and yet there is still a gap between abstract mathematical foundations and concrete applications to machine learning tasks. In this paper we introduce DisCoPyro as a categorical structure learning framework, which combines categorical structures (such as symmetric monoidal categories and operads) with amortized variational inference, and can be applied, e.g., in program learning for variational autoencoders. We provide both mathematical foundations and concrete applications together with comparison of experimental performance with other models (e.g., neuro-symbolic models). We speculate that DisCoPyro could ultimately contribute to the development of artificial general intelligence.