Sinusoidal Flow: A Fast Invertible Autoregressive Flow

TL;DR

Sinusoidal Flow introduces a novel normalising flow model that combines expressiveness, fast invertibility, and exact Jacobian computation, enabling efficient modeling and sampling of complex distributions.

Contribution

It proposes Sinusoidal Flow, a new autoregressive flow that guarantees fast invertibility without sequential inversion, balancing expressiveness and computational efficiency.

Findings

Able to model complex distributions effectively

Reliable inversion even with many layers

Generates realistic samples efficiently

Abstract

Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.