Studying number theory with deep learning: a case study with the M\"obius and squarefree indicator functions

TL;DR

This paper explores the application of deep learning, specifically small transformer models, to predict number-theoretic functions like the Möbius function and squarefree indicator, providing insights into their structure.

Contribution

It demonstrates that transformer models can learn and predict number-theoretic functions, offering a novel intersection of deep learning and number theory with theoretical explanations.

Findings

Transformers achieve nontrivial accuracy in predicting μ(n) and μ²(n)

Model analysis offers theoretical insights into number-theoretic functions

Deep learning models reveal structural properties of number theory functions

Abstract

Building on work of Charton, we train small transformer models to calculate the M\"{o}bius function $\mu(n)$ and the squarefree indicator function $\mu^2(n)$. The models attain nontrivial predictive power. We apply a mixture of additional models and feature scoring to give a theoretical explanation.