Homology-constrained vector quantization entropy regularizer

TL;DR

This paper introduces a homology-constrained entropy regularizer for vector quantization in VQ-VAEs, which increases codebook utilization and preserves latent space topology by analyzing persistent homology diagrams.

Contribution

It proposes a novel regularization method based on persistent homology to improve VQ embeddings, enhancing entropy and topology preservation in latent spaces.

Findings

Increased embedding entropy correlates with better codebook utilization.

Homology-constrained regularization mitigates overfitting and codebook collapse.

The method is implemented and tested within a VQ-VAE framework.

Abstract

This paper describes an entropy regularization term for vector quantization (VQ) based on the analysis of persistent homology of the VQ embeddings. Higher embedding entropy positively correlates with higher codebook utilization, mitigating overfit towards the identity and codebook collapse in VQ-based autoencoders [1]. We show that homology-constrained regularization is an effective way to increase entropy of the VQ process (approximated to input entropy) while preserving the approximated topology in the quantized latent space, averaged over mini batches. This work further explores some patterns of persistent homology diagrams of latents formed by vector quantization. We implement and test the proposed algorithm as a module integrated into a sample VQ-VAE. Linked code repository provides a functioning implementation of the proposed architecture, referred to as homology-constrained vector quantization (HC-VQ) further in this work.