Multiple Invertible and Partial-Equivariant Function for Latent Vector Transformation to Enhance Disentanglement in VAEs

TL;DR

This paper introduces MIPE-Transformation, a novel method that enhances disentanglement in VAEs by combining invertible, partial-equivariant transformations with exponential-family prior extension, leading to improved performance on multiple datasets.

Contribution

The paper proposes MIPE-Transformation, integrating invertible partial-equivariant mappings and exponential-family priors to improve disentanglement in VAEs, a novel combination not previously explored.

Findings

Improved disentanglement scores on 3D Cars, 3D Shapes, and dSprites datasets.

Demonstrates the effectiveness of invertible partial-equivariant transformations in VAEs.

Extends Gaussian prior to exponential family for better latent representation.

Abstract

Disentanglement learning is central to understanding and reusing learned representations in variational autoencoders (VAEs). Although equivariance has been explored in this context, effectively exploiting it for disentanglement remains challenging. In this paper, we propose a novel method, called Multiple Invertible and Partial-Equivariant Transformation (MIPE-Transformation), which integrates two main parts: (1) Invertible and Partial-Equivariant Transformation (IPE-Transformation), guaranteeing an invertible latent-to-transformed-latent mapping while preserving partial input-to-latent equivariance in the transformed latent space; and (2) Exponential-Family Conversion (EF-Conversion) to extend the standard Gaussian prior to an approximate exponential family via a learnable conversion. In experiments on the 3D Cars, 3D Shapes, and dSprites datasets, MIPE-Transformation improves the disentanglement performance of state-of-the-art VAEs.