Toward a Functional Geometric Algebra for Natural Language Semantics

TL;DR

This paper proposes using geometric algebra, specifically Clifford algebras, as a mathematically superior foundation for natural language semantics, enhancing interpretability and compositionality in neural models.

Contribution

It introduces a Functional Geometric Algebra framework that extends geometric algebra for typed, compositional semantics compatible with neural architectures.

Findings

GA provides three core capabilities absent in linear algebra.

A detailed example illustrates operator-level semantic contrasts.

GA-based operations can be integrated into transformer architectures.

Abstract

Distributional and neural approaches to natural language semantics have been built almost exclusively on conventional linear algebra: vectors, matrices, tensors, and the operations that accompany them. These methods have achieved remarkable empirical success, yet they face persistent structural limitations in compositional semantics, type sensitivity, and interpretability. I argue in this paper that geometric algebra (GA) -- specifically, Clifford algebras -- provides a mathematically superior foundation for semantic representation, and that a Functional Geometric Algebra (FGA) framework extends GA toward a typed, compositional semantics capable of supporting inference, transformation, and interpretability while retaining full compatibility with distributional learning and modern neural architectures. I develop the formal foundations, identify three core capabilities that GA provides and linear algebra does not, present a detailed worked example illustrating operator-level semantic contrasts, and show how GA-based operations already implicit in current transformer architectures can be made explicit and extended. The central claim is not merely increased dimensionality but increased structural organization: GA expands an $n$-dimensional embedding space into a $2^n$ multivector algebra where base semantic concepts and their higher-order interactions are represented within a single, principled algebraic framework.