Transport of Algebraic Structure to Latent Embeddings

TL;DR

This paper introduces a method to transfer algebraic structures from input spaces to their latent embeddings, enabling consistent operations like union while respecting mathematical laws, improving the accuracy of learned set operations.

Contribution

It proposes a general procedure to parameterize latent space operations that are consistent with input space laws by learning a bijection to a mirrored algebra.

Findings

Structural transport nets outperform baselines in respecting algebraic laws.

Respecting algebraic structure improves accuracy of set operations.

Method is applicable to various algebraic structures in latent spaces.

Abstract

Machine learning often aims to produce latent embeddings of inputs which lie in a larger, abstract mathematical space. For example, in the field of 3D modeling, subsets of Euclidean space can be embedded as vectors using implicit neural representations. Such subsets also have a natural algebraic structure including operations (e.g., union) and corresponding laws (e.g., associativity). How can we learn to "union" two sets using only their latent embeddings while respecting associativity? We propose a general procedure for parameterizing latent space operations that are provably consistent with the laws on the input space. This is achieved by learning a bijection from the latent space to a carefully designed mirrored algebra which is constructed on Euclidean space in accordance with desired laws. We evaluate these structural transport nets for a range of mirrored algebras against baselines that operate directly on the latent space. Our experiments provide strong evidence that respecting the underlying algebraic structure of the input space is key for learning accurate and self-consistent operations.