LEMMA: Bootstrapping High-Level Mathematical Reasoning with Learned Symbolic Abstractions

TL;DR

LEMMA introduces a hierarchical abstraction method for reinforcement learning agents to improve mathematical reasoning, enabling better problem-solving and generalization in tasks like equation solving and fraction simplification.

Contribution

It proposes a novel abstraction step in Expert Iteration that enhances reinforcement learning for mathematical reasoning tasks.

Findings

LEMMA improves problem-solving success rates.

Agents generalize better to harder problems.

Effective in tasks like equation solving and fraction simplification.

Abstract

Humans tame the complexity of mathematical reasoning by developing hierarchies of abstractions. With proper abstractions, solutions to hard problems can be expressed concisely, thus making them more likely to be found. In this paper, we propose Learning Mathematical Abstractions (LEMMA): an algorithm that implements this idea for reinforcement learning agents in mathematical domains. LEMMA augments Expert Iteration with an abstraction step, where solutions found so far are revisited and rewritten in terms of new higher-level actions, which then become available to solve new problems. We evaluate LEMMA on two mathematical reasoning tasks--equation solving and fraction simplification--in a step-by-step fashion. In these two domains, LEMMA improves the ability of an existing agent, both solving more problems and generalizing more effectively to harder problems than those seen during training.