The Uniformly Rotated Mondrian Kernel

TL;DR

This paper introduces a rotation-invariant kernel approximation using a uniformly rotated Mondrian process, providing theoretical analysis and demonstrating improved empirical performance on rotated datasets.

Contribution

It proposes a novel rotation-invariant kernel approximation method based on uniformly rotated Mondrian processes, with theoretical convergence guarantees and empirical validation.

Findings

Closed-form expression for the isotropic kernel approximation.

Uniform convergence rate of the rotated Mondrian kernel.

Improved empirical performance on rotated datasets.

Abstract

Random feature maps are used to decrease the computational cost of kernel machines in large-scale problems. The Mondrian kernel is one such example of a fast random feature approximation of the Laplace kernel, generated by a computationally efficient hierarchical random partition of the input space known as the Mondrian process. In this work, we study a variation of this random feature map by applying a uniform random rotation to the input space before running the Mondrian process to approximate a kernel that is invariant under rotations. We obtain a closed-form expression for the isotropic kernel that is approximated, as well as a uniform convergence rate of the uniformly rotated Mondrian kernel to this limit. To this end, we utilize techniques from the theory of stationary random tessellations in stochastic geometry and prove a new result on the geometry of the typical cell of the superposition of uniformly rotated Mondrian tessellations. Finally, we test the empirical performance of this random feature map on both synthetic and real-world datasets, demonstrating its improved performance over the Mondrian kernel on a dataset that is debiased from the standard coordinate axes.