Stepwise Variational Inference with Vine Copulas

TL;DR

This paper introduces a novel stepwise variational inference method using vine copulas, enabling flexible modeling of complex latent dependencies with an adaptive structure, outperforming mean-field VI in sparse Gaussian processes.

Contribution

The paper presents a new stepwise estimation procedure for vine copula-based variational inference, incorporating Rényi divergence and an adaptive stopping criterion to balance complexity and performance.

Findings

Outperforms mean-field VI in sparse Gaussian processes

Uses Rényi divergence for better parameter recovery

Adaptive stopping criterion simplifies model complexity

Abstract

We propose stepwise variational inference (VI) with vine copulas: a universal VI procedure that combines vine copulas with a novel stepwise estimation procedure of the variational parameters. Vine copulas consist of a nested sequence of trees built from copulas, where more complex latent dependence can be modeled with increasing number of trees. We propose to estimate the vine copula approximate posterior in a stepwise fashion, tree by tree along the vine structure. Further, we show that the usual backward Kullback-Leibler divergence cannot recover the correct parameters in the vine copula model, thus the evidence lower bound is defined based on the R\'enyi divergence. Finally, an intuitive stopping criterion for adding further trees to the vine eliminates the need to pre-define a complexity parameter of the variational distribution, as required for most other approaches. Thus, our method interpolates between mean-field VI (MFVI) and full latent dependence. In many applications, in particular sparse Gaussian processes, our method is parsimonious with parameters, while outperforming MFVI.