D-VAE: A Variational Autoencoder for Directed Acyclic Graphs

TL;DR

This paper introduces D-VAE, a novel variational autoencoder designed for directed acyclic graphs, leveraging graph neural networks and asynchronous message passing to generate and optimize DAGs for applications like neural architecture search and Bayesian network learning.

Contribution

The paper presents a new DAG-specific variational autoencoder with an asynchronous message passing scheme, enabling effective encoding, generation, and optimization of DAGs.

Findings

D-VAE can generate valid and novel DAGs.

The model produces a smooth latent space for efficient search.

D-VAE improves neural architecture search and Bayesian network learning.

Abstract

Graph structured data are abundant in the real world. Among different graph types, directed acyclic graphs (DAGs) are of particular interest to machine learning researchers, as many machine learning models are realized as computations on DAGs, including neural networks and Bayesian networks. In this paper, we study deep generative models for DAGs, and propose a novel DAG variational autoencoder (D-VAE). To encode DAGs into the latent space, we leverage graph neural networks. We propose an asynchronous message passing scheme that allows encoding the computations on DAGs, rather than using existing simultaneous message passing schemes to encode local graph structures. We demonstrate the effectiveness of our proposed DVAE through two tasks: neural architecture search and Bayesian network structure learning. Experiments show that our model not only generates novel and valid DAGs, but also produces a smooth latent space that facilitates searching for DAGs with better performance through Bayesian optimization.