Multi-Method Self-Training: Improving Code Generation With Text, And Vice Versa

TL;DR

This paper introduces Multi-Method Self-Training (MMST), a novel approach where different methods are used to train each other, enhancing performance and robustness in large language models for code and text tasks.

Contribution

The paper presents MMST, a new training technique that leverages multiple methods to improve model performance and task versatility, demonstrated on a 176B parameter model.

Findings

Improves less performant methods by up to 30%.

Enhances more performant methods by up to 32.2%.

Boosts related task performance by up to 10.3%.

Abstract

Large Language Models have many methods for solving the same problem. This introduces novel strengths (different methods may work well for different problems) and weaknesses (it may be difficult for users to know which method to use). In this paper, we introduce Multi-Method Self-Training (MMST), where one method is trained on the filtered outputs of another, allowing us to augment the strengths and ameliorate the weaknesses of each method. Using a 176B parameter model trained on both language and code, we show that MMST can 1) improve the less performant method (up to 30%) making the model easier to use, 2) improve the more performant method (up to 32.2%) making the model more performant, and 3) improve the performance of related but distinct tasks (up to 10.3%) by improving the ability of the model to generate rationales. We then conduct ablation analyses to explore why MMST works. We show that MMST generates more data than traditional self-training, but the improvement in performance is driven by the use of multiple methods. We also analyze prompt-engineering and anti-correlated performance between methods as means of making MMST more effective. We hope the evidence from our paper motivates machine learning researchers to explore ways in which advances in language models allow for new forms of training.