On periodic distributed representations using Fourier embeddings

TL;DR

This paper explores the construction and formalization of periodic high-dimensional embeddings, like Dirichlet and Gaussian kernels, to improve processing of angular data and control similarity measures.

Contribution

It formalizes the use of Fourier-based periodic embeddings within the Spatial Semantic Pointers framework, enhancing representation of angular signals.

Findings

Periodic embeddings improve processing of angular data.

The formalization enables flexible control over similarity measures.

Constructed kernels include Dirichlet and Gaussian types.

Abstract

Periodic signals are critical for representing physical and perceptual phenomena. Scalar, real angular measures, e.g., radians and degrees, result in difficulty processing and distinguishing nearby angles, especially when their absolute difference exceeds pi. We can avoid this problem by using real-valued, periodic embeddings in high-dimensional space. These representations also allow us to control the nature of their dot product similarities, allowing us to construct a variety of different kernel shapes. In this work, we aim of highlight how these representations can be constructed and focus on the formalization of Dirichlet and periodic Gaussian kernels using the neurally-plausible representation scheme of Spatial Semantic Pointers.