4OPS: Structural Difficulty Modeling in Integer Arithmetic Puzzles

TL;DR

This paper introduces a structural difficulty model for integer arithmetic puzzles, using an exact solver to analyze features that determine puzzle difficulty, enabling explainable and scalable difficulty estimation.

Contribution

It develops an exact dynamic programming solver and a large dataset to identify structural attributes that fully determine puzzle difficulty, advancing explainable difficulty modeling.

Findings

Difficulty correlates with minimal construction size.

Baseline models struggle to distinguish easy puzzles.

Structural features fully explain puzzle difficulty.

Abstract

Arithmetic puzzle games provide a controlled setting for studying difficulty in mathematical reasoning tasks, a core challenge in adaptive learning systems. We investigate the structural determinants of difficulty in a class of integer arithmetic puzzles inspired by number games. We formalize the problem and develop an exact dynamic-programming solver that enumerates reachable targets, extracts minimal-operation witnesses, and enables large-scale labeling. Using this solver, we construct a dataset of over 3.4 million instances and define difficulty via the minimum number of operations required to reach a target. We analyze the relationship between difficulty and solver-derived features. While baseline machine learning models based on bag- and target-level statistics can partially predict solvability, they fail to reliably distinguish easy instances. In contrast, we show that difficulty is fully determined by a small set of interpretable structural attributes derived from exact witnesses. In particular, the number of input values used in a minimal construction serves as a minimal sufficient statistic for difficulty under this labeling. These results provide a transparent, computationally grounded account of puzzle difficulty that bridges symbolic reasoning and data-driven modeling. The framework supports explainable difficulty estimation and principled task sequencing, with direct implications for adaptive arithmetic learning and intelligent practice systems.