PROFIT: A Specialized Optimizer for Deep Fine Tuning

TL;DR

PROFIT is a novel optimizer specifically designed for fine-tuning pre-trained models, improving performance across diverse tasks by leveraging properties of converged models and employing a temporal gradient-orthogonalization process.

Contribution

It introduces PROFIT, a specialized optimizer for incremental fine-tuning that explicitly considers converged model properties, outperforming traditional optimizers in various applications.

Findings

PROFIT outperforms traditional optimizers in multiple tasks.

It effectively regularizes the fine-tuning process for better performance.

Easy integration into existing training pipelines.

Abstract

The fine-tuning of pre-trained models has become ubiquitous in generative AI, computer vision, and robotics. Although much attention has been paid to improving the efficiency of fine-tuning model, there has been less scholarship around fine-tuning specifically for improved model performance. To remedy this gap, we present PROFIT, one of the first optimizers designed to incrementally fine-tune converged models on new tasks and/or datasets. Unlike traditional optimizers such as SGD or Adam, which make minimal assumptions due to random initializations, PROFIT takes the properties of a converged model into account explicitly to regularize the optimization process. Employing a temporal gradient-orthogonalization process, PROFIT outperforms fine-tuning methods in various tasks, from image classification to multimodal language model training to large-scale motion prediction. Moreover, PROFIT is encapsulated as a modular optimizer, which makes it easy to integrate directly into any training pipeline with minimal engineering effort.