A Simplex Witness Certificate for Constant Collapse in Variational Autoencoders

TL;DR

This paper introduces a theoretical framework and practical protocol to certify when variational autoencoders do not suffer from input-independent collapse, using a fixed simplex witness and teacher posterior construction.

Contribution

It proposes a novel certificate for exact constant collapse in VAEs, along with a minimal training protocol and theoretical insights for preventing collapse.

Findings

Constructed a fixed teacher posterior using GMM approximation of data.

Developed an alignment loss baseline for detecting collapse.

Provided preliminary MNIST results supporting the method.

Abstract

We study exact constant collapse in variational autoencoders, where the deterministic encoder path becomes independent of the input. The VAE prior is kept as the standard Gaussian. Before VAE training, we construct a single fixed teacher posterior by searching a GMM-based approximation of the data. We then attach a fixed latent-only simplex witness to the encoder mean and compare its output with the teacher. The resulting alignment loss has an exact constant-predictor baseline: if the latent witness beats this baseline, the encoder mean cannot be input-independent constant. The same construction also gives a closed-form latent target that realizes zero teacher-witness alignment error for any full-support teacher posterior. This yields a concrete design principle: choose a teacher with nontrivial information but controlled log-odds energy, fix the witness, train only the encoder and decoder, and certify non-collapse by a positive margin. We present the theory, a minimal training protocol, and preliminary MNIST sanity checks. The analysis targets exact constant collapse. Reconstruction quality, sampling quality, and other collapse modes are evaluated with additional diagnostics rather than folded into the certificate itself.