COMET: "Cone of experience" enhanced large multimodal model for mathematical problem generation

TL;DR

This paper introduces COMET, a multimodal model for mathematical problem generation that unifies problem solving and generation, guided by a 'Cone of Experience' framework, and demonstrates its effectiveness on Chinese datasets.

Contribution

The paper proposes a novel 'Cone of Experience' guided fine-tuning framework and a unified approach for problem generation and solving in multimodal models.

Findings

Effective problem generation demonstrated on Chinese datasets.

Unified framework improves model versatility and performance.

Constructed a new Chinese multimodal mathematical problem dataset.

Abstract

The automatic generation of high-quality mathematical problems is practically valuable in many educational scenarios. Large multimodal model provides a novel technical approach for the mathematical problem generation because of its wide success in cross-modal data scenarios. However, the traditional method of separating problem solving from problem generation and the mainstream fine-tuning framework of monotonous data structure with homogeneous training objectives limit the application of large multimodal model in mathematical problem generation. Addressing these challenges, this paper proposes COMET, a "Cone of Experience" enhanced large multimodal model for mathematical problem generation. Firstly, from the perspective of mutual ability promotion and application logic, we unify stem generation and problem solving into mathematical problem generation. Secondly, a three-stage fine-turning framework guided by the "Cone of Experience" is proposed. The framework divides the fine-tuning data into symbolic experience, iconic experience, and direct experience to draw parallels with experiences in the career growth of teachers. Several fine-grained data construction and injection methods are designed in this framework. Finally, we construct a Chinese multimodal mathematical problem dataset to fill the vacancy of Chinese multimodal data in this field. Combined with objective and subjective indicators, experiments on multiple datasets fully verify the effectiveness of the proposed framework and model.