McTorch, a manifold optimization library for deep learning

TL;DR

McTorch is a library that integrates manifold optimization into PyTorch, enabling deep learning models to incorporate manifold constraints like orthogonality and rank with ease.

Contribution

It introduces a flexible, PyTorch-compatible library that simplifies the use of manifold constraints in deep learning models.

Findings

Enables easy integration of manifold constraints in deep learning.

Decouples manifold definitions from optimizers for flexibility.

Provides open-source implementation for community use.

Abstract

In this paper, we introduce McTorch, a manifold optimization library for deep learning that extends PyTorch. It aims to lower the barrier for users wishing to use manifold constraints in deep learning applications, i.e., when the parameters are constrained to lie on a manifold. Such constraints include the popular orthogonality and rank constraints, and have been recently used in a number of applications in deep learning. McTorch follows PyTorch's architecture and decouples manifold definitions and optimizers, i.e., once a new manifold is added it can be used with any existing optimizer and vice-versa. McTorch is available at https://github.com/mctorch .