CF-GO-Net: A Universal Distribution Learner via Characteristic Function Networks with Graph Optimizers

TL;DR

CF-GO-Net introduces a flexible, distribution-agnostic generative modeling approach using characteristic functions and graph neural network optimizers, enabling effective learning directly in feature spaces and broad applicability.

Contribution

It presents a novel distribution learning method leveraging characteristic functions and a GNN-based optimizer, removing pdf constraints and enabling learning in pre-trained model feature spaces.

Findings

Effective distribution learning without pdf assumptions

Flexible sampling strategy via GNN optimizer

Capable of learning directly in feature spaces

Abstract

Generative models aim to learn the distribution of datasets, such as images, so as to be able to generate samples that statistically resemble real data. However, learning the underlying probability distribution can be very challenging and intractable. To this end, we introduce an approach which employs the characteristic function (CF), a probabilistic descriptor that directly corresponds to the distribution. However, unlike the probability density function (pdf), the characteristic function not only always exists, but also provides an additional degree of freedom, hence enhances flexibility in learning distributions. This removes the critical dependence on pdf-based assumptions, which limit the applicability of traditional methods. While several works have attempted to use CF in generative modeling, they often impose strong constraints on the training process. In contrast, our approach calculates the distance between query points in the CF domain, which is an unconstrained and well defined problem. Next, to deal with the sampling strategy, which is crucial to model performance, we propose a graph neural network (GNN)-based optimizer for the sampling process, which identifies regions where the difference between CFs is most significant. In addition, our method allows the use of a pre-trained model, such as a well-trained autoencoder, and is capable of learning directly in its feature space, without modifying its parameters. This offers a flexible and robust approach to generative modeling, not only provides broader applicability and improved performance, but also equips any latent space world with the ability to become a generative model.