AI capabilities can be significantly improved without expensive retraining

TL;DR

Post-training enhancements significantly boost AI performance without costly retraining, offering a cost-effective way to improve capabilities across various tasks.

Contribution

The paper categorizes recent post-training enhancement techniques and quantifies their benefits in terms of compute-equivalent gains.

Findings

Most enhancements improve performance by over 5x in compute terms

Some enhancements achieve more than 20x improvements

Fine-tuning costs are typically less than 1% of original training costs

Abstract

State-of-the-art AI systems can be significantly improved without expensive retraining via "post-training enhancements"-techniques applied after initial training like fine-tuning the system to use a web browser. We review recent post-training enhancements, categorizing them into five types: tool-use, prompting methods, scaffolding, solution selection, and data generation. Different enhancements improve performance on different tasks, making it hard to compare their significance. So we translate improvements from different enhancements into a common currency, the compute-equivalent gain: how much additional training compute would be needed to improve performance by the same amount as the enhancement. Our non-experimental work shows that post-training enhancements have significant benefits: most surveyed enhancements improve benchmark performance by more than a 5x increase in training compute, some by more than 20x. Post-training enhancements are relatively cheap to develop: fine-tuning costs are typically <1% of the original training cost. Governing the development of capable post-training enhancements may be challenging because frontier models could be enhanced by a wide range of actors.