Learning Functors using Gradient Descent

TL;DR

This paper introduces a category-theoretic framework for neural networks, specifically applying it to CycleGAN, enabling the learning of functors via gradient descent for unpaired image translation.

Contribution

It formalizes neural networks as functors in a categorical setting and extends learning to arbitrary categories, introducing a novel system for unpaired image editing.

Findings

Successfully learned object insertion and deletion without paired data

Demonstrated promising qualitative results on CelebA dataset

Generalized cycle-consistency as composition invariants in a categorical framework

Abstract

Neural networks are a general framework for differentiable optimization which includes many other machine learning approaches as special cases. In this paper we build a category-theoretic formalism around a neural network system called CycleGAN. CycleGAN is a general approach to unpaired image-to-image translation that has been getting attention in the recent years. Inspired by categorical database systems, we show that CycleGAN is a "schema", i.e. a specific category presented by generators and relations, whose specific parameter instantiations are just set-valued functors on this schema. We show that enforcing cycle-consistencies amounts to enforcing composition invariants in this category. We generalize the learning procedure to arbitrary such categories and show a special class of functors, rather than functions, can be learned using gradient descent. Using this framework we design a novel neural network system capable of learning to insert and delete objects from images without paired data. We qualitatively evaluate the system on the CelebA dataset and obtain promising results.