Teaching Autoregressive Language Models Complex Tasks By Demonstration

TL;DR

Fine-tuning GPT-Neo with structured demonstrations enables it to perform complex mathematical tasks like longhand modulo operations with high accuracy, even with limited training data.

Contribution

This work shows that small, well-structured demonstration datasets can significantly improve autoregressive models' ability to perform complex tasks without changing the underlying learning algorithm.

Findings

GPT-Neo achieves over 80% accuracy on long division tasks after fine-tuning.

Structured demonstrations drastically improve model performance on complex multi-step tasks.

Small datasets of demonstrations can teach models complex skills without extensive retraining.

Abstract

This paper demonstrates that by fine-tuning an autoregressive language model (GPT-Neo) on appropriately structured step-by-step demonstrations, it is possible to teach it to execute a mathematical task that has previously proved difficult for Transformers - longhand modulo operations - with a relatively small number of examples. Specifically, we fine-tune GPT-Neo to solve the numbers__div_remainder task from the DeepMind Mathematics Dataset; Saxton et al. (arXiv:1904.01557) reported below 40% accuracy on this task with 2 million training examples. We show that after fine-tuning on 200 appropriately structured demonstrations of solving long division problems and reporting the remainders, the smallest available GPT-Neo model achieves over 80% accuracy. This is achieved by constructing an appropriate dataset for fine-tuning, with no changes to the learning algorithm. These results suggest that fine-tuning autoregressive language models on small sets of well-crafted demonstrations may be a useful paradigm for enabling individuals without training in machine learning to coax such models to perform some kinds of complex multi-step tasks.