MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents

TL;DR

This paper presents MathViz-E, an automated math visualization and solving system that uses domain-specific tools, along with datasets and evaluation methods, to enhance AI's ability to assist in mathematical education.

Contribution

It introduces a domain-specific system for math visualization and solving, along with datasets and an auto-evaluator, addressing challenges in tool control and system evaluation.

Findings

Created specialized datasets for math visualization and solving.

Developed an auto-evaluator for system output comparison.

Open sourced datasets and code for community use.

Abstract

There has been significant recent interest in harnessing LLMs to control software systems through multi-step reasoning, planning and tool-usage. While some promising results have been obtained, application to specific domains raises several general issues including the control of specialized domain tools, the lack of existing datasets for training and evaluation, and the non-triviality of automated system evaluation and improvement. In this paper, we present a case-study where we examine these issues in the context of a specific domain. Specifically, we present an automated math visualizer and solver system for mathematical pedagogy. The system orchestrates mathematical solvers and math graphing tools to produce accurate visualizations from simple natural language commands. We describe the creation of specialized data-sets, and also develop an auto-evaluator to easily evaluate the outputs of our system by comparing them to ground-truth expressions. We have open sourced the data-sets and code for the proposed system.