Quasi-Autoregressive Residual (QuAR) Flows

TL;DR

This paper introduces Quasi-Autoregressive Residual (QuAR) Flows, a simplified residual flow method that significantly reduces computational costs while maintaining modeling benefits, enabling broader practical use of flow-based models.

Contribution

The paper proposes a Quasi-Autoregressive approach to residual flows, reducing complexity and resource requirements while preserving their modeling capabilities.

Findings

Drastically reduced training and inference time.

Maintained high modeling accuracy with lower computational cost.

Broader applicability of flow-based models in practice.

Abstract

Normalizing Flows are a powerful technique for learning and modeling probability distributions given samples from those distributions. The current state of the art results are built upon residual flows as these can model a larger hypothesis space than coupling layers. However, residual flows are extremely computationally expensive both to train and to use, which limits their applicability in practice. In this paper, we introduce a simplification to residual flows using a Quasi-Autoregressive (QuAR) approach. Compared to the standard residual flow approach, this simplification retains many of the benefits of residual flows while dramatically reducing the compute time and memory requirements, thus making flow-based modeling approaches far more tractable and broadening their potential applicability.