A Deep Generative Approach to Stratified Learning

TL;DR

This paper introduces two deep generative frameworks for modeling and learning distributions on stratified spaces, addressing challenges posed by varying dimensions and singularities.

Contribution

It develops a dimension-aware mixture of VAEs and a diffusion-based method, with theoretical convergence and consistency guarantees for stratified learning.

Findings

Convergence rates depend on intrinsic dimensions and smoothness.

Algorithms accurately estimate the number and dimensions of strata.

Methods outperform existing models in molecular dynamics applications.

Abstract

While the manifold hypothesis is widely adopted in modern machine learning, complex data is often better modeled as stratified spaces -- unions of manifolds (strata) of varying dimensions. Stratified learning is challenging due to varying dimensionality, intersection singularities, and lack of efficient models in learning the underlying distributions. We provide a deep generative approach to stratified learning by developing two generative frameworks for learning distributions on stratified spaces. The first is a sieve maximum likelihood approach realized via a dimension-aware mixture of variational autoencoders. The second is a diffusion-based framework that explores the score field structure of a mixture. We establish the convergence rates for learning both the ambient and intrinsic distributions, which are shown to be dependent on the intrinsic dimensions and smoothness of the underlying strata. Utilizing the geometry of the score field, we also establish consistency for estimating the intrinsic dimension of each stratum and propose an algorithm that consistently estimates both the number of strata and their dimensions. Theoretical results for both frameworks provide fundamental insights into the interplay of the underlying geometry, the ambient noise level, and deep generative models. Extensive simulations and real dataset applications, such as molecular dynamics, demonstrate the effectiveness of our methods.