# Knowledge-Based Regularization in Generative Modeling

## TL;DR

This paper introduces a regularization technique that integrates prior knowledge of feature relations into general-purpose generative models, improving their learning process without altering existing architectures.

## Contribution

It proposes a flexible regularizer that encodes feature relation knowledge into various generative models using standard backpropagation, enhancing their ability to incorporate domain expertise.

## Key findings

- Effective across multiple datasets and models
- Compatible with VAEs and GANs
- Improves generative modeling with prior knowledge

## Abstract

Prior domain knowledge can greatly help to learn generative models. However, it is often too costly to hard-code prior knowledge as a specific model architecture, so we often have to use general-purpose models. In this paper, we propose a method to incorporate prior knowledge of feature relations into the learning of general-purpose generative models. To this end, we formulate a regularizer that makes the marginals of a generative model to follow prescribed relative dependence of features. It can be incorporated into off-the-shelf learning methods of many generative models, including variational autoencoders and generative adversarial networks, as its gradients can be computed using standard backpropagation techniques. We show the effectiveness of the proposed method with experiments on multiple types of datasets and generative models.