Conditional Permutation Invariant Flows

TL;DR

This paper introduces a new permutation-invariant flow model for set data that effectively generates complex, conditioned outputs like traffic scenes and object bounding boxes, outperforming existing methods.

Contribution

It proposes a novel continuous normalizing flow with permutation-equivariant dynamics driven by neural networks, enabling efficient and realistic set data generation conditioned on auxiliary information.

Findings

Outperforms non-permutation invariant baselines in log likelihood

Generates realistic traffic scenes conditioned on maps

Produces accurate object bounding boxes conditioned on images

Abstract

We present a novel, conditional generative probabilistic model of set-valued data with a tractable log density. This model is a continuous normalizing flow governed by permutation equivariant dynamics. These dynamics are driven by a learnable per-set-element term and pairwise interactions, both parametrized by deep neural networks. We illustrate the utility of this model via applications including (1) complex traffic scene generation conditioned on visually specified map information, and (2) object bounding box generation conditioned directly on images. We train our model by maximizing the expected likelihood of labeled conditional data under our flow, with the aid of a penalty that ensures the dynamics are smooth and hence efficiently solvable. Our method significantly outperforms non-permutation invariant baselines in terms of log likelihood and domain-specific metrics (offroad, collision, and combined infractions), yielding realistic samples that are difficult to distinguish from real data.