A short letter on the dot product between rotated Fourier transforms

TL;DR

This paper derives a new mathematical formula linking spatial displacement to similarity in Spatial Semantic Pointers, challenging previous Gaussian conjectures and providing a foundation for neural network design involving spatial structures.

Contribution

It introduces a simple trigonometric formula for SSP similarity and proves the expected similarity as a product of sinc functions under certain conditions.

Findings

Expected similarity is a product of sinc functions of displacement.

Contradicts previous Gaussian similarity conjecture.

Provides a mathematical link between space and SSP similarity.

Abstract

Spatial Semantic Pointers (SSPs) have recently emerged as a powerful tool for representing and transforming continuous space, with numerous applications to cognitive modelling and deep learning. Fundamental to SSPs is the notion of "similarity" between vectors representing different points in $n$-dimensional space -- typically the dot product or cosine similarity between vectors with rotated unit-length complex coefficients in the Fourier domain. The similarity measure has previously been conjectured to be a Gaussian function of Euclidean distance. Contrary to this conjecture, we derive a simple trigonometric formula relating spatial displacement to similarity, and prove that, in the case where the Fourier coefficients are uniform i.i.d., the expected similarity is a product of normalized sinc functions: $\prod_{k=1}^{n} \operatorname{sinc} \left( a_k \right)$, where $\mathbf{a} \in \mathbb{R}^n$ is the spatial displacement between the two $n$-dimensional points. This establishes a direct link between space and the similarity of SSPs, which in turn helps bolster a useful mathematical framework for architecting neural networks that manipulate spatial structures.