Randomized Geometric Algebra Methods for Convex Neural Networks

TL;DR

This paper introduces randomized algorithms within Clifford's Geometric Algebra to enable convex optimization in neural networks and transfer learning, improving robustness and performance of large language model embeddings.

Contribution

It presents a novel intersection of geometric algebra and convex optimization for neural networks and transfer learning, with practical applications to LLM embeddings.

Findings

Convex optimization via geometric algebra improves LLM embedding performance.

Enhanced robustness and stability in transfer learning across multiple datasets.

Demonstrated effectiveness on GPT-4 and BERT embeddings with various hyperparameters.

Abstract

We introduce randomized algorithms to Clifford's Geometric Algebra, generalizing randomized linear algebra to hypercomplex vector spaces. This novel approach has many implications in machine learning, including training neural networks to global optimality via convex optimization. Additionally, we consider fine-tuning large language model (LLM) embeddings as a key application area, exploring the intersection of geometric algebra and modern AI techniques. In particular, we conduct a comparative analysis of the robustness of transfer learning via embeddings, such as OpenAI GPT models and BERT, using traditional methods versus our novel approach based on convex optimization. We test our convex optimization transfer learning method across a variety of case studies, employing different embeddings (GPT-4 and BERT embeddings) and different text classification datasets (IMDb, Amazon Polarity Dataset, and GLUE) with a range of hyperparameter settings. Our results demonstrate that convex optimization and geometric algebra not only enhances the performance of LLMs but also offers a more stable and reliable method of transfer learning via embeddings.