Masked Conditioning for Deep Generative Models

TL;DR

This paper presents a masked-conditioning technique for deep generative models that effectively handles sparse, mixed-type data in engineering applications, improving generation quality with limited data and resources.

Contribution

The authors introduce a novel masked-conditioning method and flexible embedding for generative models, enabling effective training on sparse, mixed-type data in engineering contexts.

Findings

Effective handling of sparse, mixed-type data in generative models.

Improved generation quality with small models and limited data.

Compatibility with large pretrained foundation models.

Abstract

Datasets in engineering domains are often small, sparsely labeled, and contain numerical as well as categorical conditions. Additionally. computational resources are typically limited in practical applications which hinders the adoption of generative models for engineering tasks. We introduce a novel masked-conditioning approach, that enables generative models to work with sparse, mixed-type data. We mask conditions during training to simulate sparse conditions at inference time. For this purpose, we explore the use of various sparsity schedules that show different strengths and weaknesses. In addition, we introduce a flexible embedding that deals with categorical as well as numerical conditions. We integrate our method into an efficient variational autoencoder as well as a latent diffusion model and demonstrate the applicability of our approach on two engineering-related datasets of 2D point clouds and images. Finally, we show that small models trained on limited data can be coupled with large pretrained foundation models to improve generation quality while retaining the controllability induced by our conditioning scheme.