Mitigating Catastrophic Forgetting in Mathematical Reasoning Finetuning through Mixed Training

TL;DR

This paper introduces mixed training strategies that interleave mathematical reasoning and natural language inference examples to prevent catastrophic forgetting in large language models during finetuning, maintaining performance across tasks.

Contribution

The study proposes and empirically validates mixed training methods that eliminate catastrophic forgetting while preserving task-specific performance in large language models.

Findings

Mixed training prevents catastrophic forgetting effectively.

Even minimal NLI exposure (6.2%) suffices for regularization.

Balanced 1:1 training ratio maintains performance across tasks.

Abstract

When finetuning large language models for specialized tasks such as mathematical reasoning, models exhibit catastrophic forgetting, losing previously learned capabilities. We investigate this by finetuning Flan-T5-Base (250M parameters) on the DeepMind Mathematics dataset and measuring forgetting on MultiNLI. Math-only training improves mathematical accuracy from 3.1\% to 12.0\% but causes NLI accuracy to collapse from 81.0\% to 16.5\%--a 64.5 percentage point drop occurring within the first 1,000 training steps. We propose mixed training strategies that interleave mathematical and NLI examples during training. Our results demonstrate that mixed training completely eliminates catastrophic forgetting while maintaining equivalent mathematical performance: the balanced 1:1 ratio achieves 12.0\% math accuracy (matching math-only) while preserving 86.2\% NLI accuracy. We systematically explore mixing ratios from 1:1 to 15:1, finding that even minimal NLI exposure (6.2\%) provides effective regularization. These findings demonstrate that specialization need not require forgetting general capabilities, with implications for scaling to larger models where mixed training may confer additional benefits beyond forgetting prevention.