Learning Mathematical Properties of Integers

TL;DR

This paper explores whether high-dimensional embeddings of integers can encode mathematical properties and improve numerical reasoning tasks, demonstrating that embeddings trained on mathematical sequences outperform those learned from text.

Contribution

It introduces a method for learning integer embeddings from mathematical data, showing they capture useful mathematical concepts for reasoning.

Findings

Integer embeddings trained on mathematical sequences outperform text-based embeddings.

Embeddings encode meaningful mathematical properties of integers.

Improved performance on numerical reasoning tasks.

Abstract

Embedding words in high-dimensional vector spaces has proven valuable in many natural language applications. In this work, we investigate whether similarly-trained embeddings of integers can capture concepts that are useful for mathematical applications. We probe the integer embeddings for mathematical knowledge, apply them to a set of numerical reasoning tasks, and show that by learning the representations from mathematical sequence data, we can substantially improve over number embeddings learned from English text corpora.