Variational Mixture of HyperGenerators for Learning Distributions Over Functions

TL;DR

VAMoH is a novel deep generative model that combines INRs, VAEs, normalizing flows, and hypernetworks to efficiently learn and perform inference over distributions of continuous functions across diverse data types.

Contribution

It introduces VAMoH, a new model integrating INRs with VAEs, normalizing flows, and hypernetworks, enabling efficient learning and inference over function distributions.

Findings

VAMoH effectively models distributions over functions for images, voxels, and climate data.

It performs inference tasks like super-resolution and in-painting better or comparable to previous methods.

VAMoH is less computationally demanding than existing approaches.

Abstract

Recent approaches build on implicit neural representations (INRs) to propose generative models over function spaces. However, they are computationally costly when dealing with inference tasks, such as missing data imputation, or directly cannot tackle them. In this work, we propose a novel deep generative model, named VAMoH. VAMoH combines the capabilities of modeling continuous functions using INRs and the inference capabilities of Variational Autoencoders (VAEs). In addition, VAMoH relies on a normalizing flow to define the prior, and a mixture of hypernetworks to parametrize the data log-likelihood. This gives VAMoH a high expressive capability and interpretability. Through experiments on a diverse range of data types, such as images, voxels, and climate data, we show that VAMoH can effectively learn rich distributions over continuous functions. Furthermore, it can perform inference-related tasks, such as conditional super-resolution generation and in-painting, as well or better than previous approaches, while being less computationally demanding.