Lattice Representation Learning

TL;DR

This paper introduces a novel approach for learning discrete lattice-based representations in Euclidean space, combining explicit lattice quantization with efficient learning algorithms, and explores their connections to Gaussian VAEs and algebraic structures.

Contribution

It presents new algorithms and theoretical insights for learning lattice representations, bridging discrete encoding with differentiable training and algebraic modeling.

Findings

Lattice representations can be learned efficiently using proposed algorithms.

A new mathematical link between training expressions and inference.

Experimental validation on two datasets demonstrates effectiveness.

Abstract

In this article we introduce theory and algorithms for learning discrete representations that take on a lattice that is embedded in an Euclidean space. Lattice representations possess an interesting combination of properties: a) they can be computed explicitly using lattice quantization, yet they can be learned efficiently using the ideas we introduce in this paper, b) they are highly related to Gaussian Variational Autoencoders, allowing designers familiar with the latter to easily produce discrete representations from their models and c) since lattices satisfy the axioms of a group, their adoption can lead into a way of learning simple algebras for modeling binary operations between objects through symbolic formalisms, yet learn these structures also formally using differentiation techniques. This article will focus on laying the groundwork for exploring and exploiting the first two properties, including a new mathematical result linking expressions used during training and inference time and experimental validation on two popular datasets.