Models Can and Should Embrace the Communicative Nature of Human-Generated Math

TL;DR

This paper argues that treating math as a form of situated linguistic communication, rather than purely symbolic, enhances AI understanding and generation of math by capturing human-like communicative intentions, as shown through experiments with language models.

Contribution

It demonstrates that language models can interpret mathematical symbols and proofs in human-like, communicative ways, emphasizing the importance of modeling math as language-based communication.

Findings

Language models interpret the equals sign in humanlike ways.

Models prefer naturalistic ordering of proofs.

Math as communication improves AI understanding.

Abstract

Math is constructed by people for people: just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models are trained on reflects not just idealized mathematical entities but rich communicative intentions. While there are important advantages to treating math in a purely symbolic manner, we here hypothesize that there are benefits to treating math as situated linguistic communication and that language models are well suited for this goal, in ways that are not fully appreciated. We illustrate these points with two case studies. First, we ran an experiment in which we found that language models interpret the equals sign in a humanlike way -- generating systematically different word problems for the same underlying equation arranged in different ways. Second, we found that language models prefer proofs to be ordered in naturalistic ways, even though other orders would be logically equivalent. We advocate for AI systems that learn from and represent the communicative intentions latent in human-generated math.