On the Convergence Rate of Gaussianization with Random Rotations

TL;DR

This paper analyzes the convergence rate of Gaussianization with random rotations, revealing a linear scaling with dimension and exploring methods to improve efficiency for high-dimensional data.

Contribution

It provides an analytical proof of the linear scaling of layers needed for Gaussianization with dimension and investigates empirical scaling behaviors for various distributions.

Findings

Number of layers scales linearly with dimension for Gaussian input

Empirical results show similar linear scaling for arbitrary distributions

Certain distributions exhibit more favorable scaling behavior

Abstract

Gaussianization is a simple generative model that can be trained without backpropagation. It has shown compelling performance on low dimensional data. As the dimension increases, however, it has been observed that the convergence speed slows down. We show analytically that the number of required layers scales linearly with the dimension for Gaussian input. We argue that this is because the model is unable to capture dependencies between dimensions. Empirically, we find the same linear increase in cost for arbitrary input $p(x)$, but observe favorable scaling for some distributions. We explore potential speed-ups and formulate challenges for further research.