Learning with the $p$-adics

TL;DR

This paper explores the use of $p$-adic numbers as an alternative mathematical framework for machine learning, offering hierarchical representations and new modeling possibilities beyond traditional real-number-based methods.

Contribution

It introduces the theoretical foundations and algorithms for learning with $p$-adics, demonstrating their potential for hierarchical and semantic network representations.

Findings

Representation of simple semantic networks as $p$-adic linear networks

Development of classification and regression models over $p$-adics

Discussion of open problems and future research directions

Abstract

Existing machine learning frameworks operate over the field of real numbers ($\mathbb{R}$) and learn representations in real (Euclidean or Hilbert) vector spaces (e.g., $\mathbb{R}^d$). Their underlying geometric properties align well with intuitive concepts such as linear separability, minimum enclosing balls, and subspace projection; and basic calculus provides a toolbox for learning through gradient-based optimization. But is this the only possible choice? In this paper, we study the suitability of a radically different field as an alternative to $\mathbb{R}$ -- the ultrametric and non-archimedean space of $p$-adic numbers, $\mathbb{Q}_p$. The hierarchical structure of the $p$-adics and their interpretation as infinite strings make them an appealing tool for code theory and hierarchical representation learning. Our exploratory theoretical work establishes the building blocks for classification, regression, and representation learning with the $p$-adics, providing learning models and algorithms. We illustrate how simple Quillian semantic networks can be represented as a compact $p$-adic linear network, a construction which is not possible with the field of reals. We finish by discussing open problems and opportunities for future research enabled by this new framework.