Sylvester Normalizing Flows for Variational Inference

TL;DR

This paper introduces Sylvester normalizing flows, a flexible and more powerful generalization of planar flows, improving variational inference by removing bottlenecks and outperforming existing flow methods on multiple datasets.

Contribution

The paper proposes Sylvester normalizing flows, a novel class of flows that enhance flexibility and performance in variational inference beyond planar and inverse autoregressive flows.

Findings

Sylvester flows outperform planar flows on several datasets.

Sylvester flows remove the bottleneck present in planar flows.

Sylvester flows demonstrate superior flexibility and effectiveness.

Abstract

Variational inference relies on flexible approximate posterior distributions. Normalizing flows provide a general recipe to construct flexible variational posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single-unit bottleneck from planar flows, making a single transformation much more flexible. We compare the performance of Sylvester normalizing flows against planar flows and inverse autoregressive flows and demonstrate that they compare favorably on several datasets.