Generative Model for Heterogeneous Inference

TL;DR

This paper introduces extended autoregressive generative models tailored for heterogeneous inference scenarios, overcoming Bayesian Network NP-hardness, with theoretical validation and superior performance on BN datasets.

Contribution

The paper develops EAR and EARA models that enable efficient heterogeneous inference, extending GMs to handle non-hierarchical, irregularly dependent variables.

Findings

EAR outperforms other GMs on BN datasets

Theoretical validation of EAR and EARA effectiveness

Successful application to Markov border inference

Abstract

Generative models (GMs) such as Generative Adversary Network (GAN) and Variational Auto-Encoder (VAE) have thrived these years and achieved high quality results in generating new samples. Especially in Computer Vision, GMs have been used in image inpainting, denoising and completion, which can be treated as the inference from observed pixels to corrupted pixels. However, images are hierarchically structured which are quite different from many real-world inference scenarios with non-hierarchical features. These inference scenarios contain heterogeneous stochastic variables and irregular mutual dependences. Traditionally they are modeled by Bayesian Network (BN). However, the learning and inference of BN model are NP-hard thus the number of stochastic variables in BN is highly constrained. In this paper, we adapt typical GMs to enable heterogeneous learning and inference in polynomial time.We also propose an extended autoregressive (EAR) model and an EAR with adversary loss (EARA) model and give theoretical results on their effectiveness. Experiments on several BN datasets show that our proposed EAR model achieves the best performance in most cases compared to other GMs. Except for black box analysis, we've also done a serial of experiments on Markov border inference of GMs for white box analysis and give theoretical results.