CDFlow: Building Invertible Layers with Circulant and Diagonal Matrices

TL;DR

This paper introduces CDFlow, a novel invertible linear layer using circulant and diagonal matrices that improves efficiency and expressiveness in normalizing flows, enabling better density estimation and faster computations.

Contribution

The authors propose a new invertible linear layer based on circulant and diagonal matrices, reducing parameter and computational complexity while maintaining expressive power.

Findings

Achieves strong density estimation on natural images.

Reduces computational complexity of matrix inversion and log-determinant calculation.

Accelerates key operations in normalizing flows for scalable generative modeling.

Abstract

Normalizing flows are deep generative models that enable efficient likelihood estimation and sampling through invertible transformations. A key challenge is to design linear layers that enhance expressiveness while maintaining efficient computation of the Jacobian determinant and inverse. We introduce a novel invertible linear layer based on the product of circulant and diagonal matrices. This decomposition reduces parameter complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$ using $m$ diagonal matrices and $m-1$ circulant matrices while still approximating general linear transformations. By leveraging the Fast Fourier Transform, our approach reduces the time complexity of matrix inversion from $\mathcal{O}(n^3)$ to $\mathcal{O}(mn\log n)$ and that of computing the log-determinant from $\mathcal{O}(n^3)$ to $\mathcal{O}(mn)$, where $n$ is the input dimension. We build upon this layer to develop Circulant-Diagonal Flow (CDFlow), which achieves strong density estimation on natural image datasets and effectively models data with inherent periodic structure. Furthermore, CDFlow significantly accelerates key operations in normalizing flows, providing practical benefits for scalable generative modeling.