Escaping Optimization Stagnation: Taking Steps Beyond Task Arithmetic via Difference Vectors

TL;DR

This paper introduces difference vectors and a novel iterative algorithm, DV-BASI, to enhance task arithmetic in model editing, overcoming optimization stagnation and improving performance across multiple tasks.

Contribution

It proposes the use of difference vectors derived from optimization history and a new iterative method, DV-BASI, to improve model editing and multi-task performance without extra modules.

Findings

DV-BASI enables continuous optimization for task arithmetic.

Models merged with DV-BASI outperform individually fine-tuned models.

The framework achieves state-of-the-art results on various benchmarks.

Abstract

Current methods for editing pre-trained models face significant challenges, primarily high computational costs and limited scalability. Task arithmetic has recently emerged as a promising solution, using simple arithmetic operations-addition and negation-based on task vectors which are the differences between fine-tuned and pre-trained model weights, to efficiently modify model behavior. However, the full potential of task arithmetic remains underexplored, primarily due to limited mechanisms for overcoming optimization stagnation. To address this challenge, we introduce the notion of difference vector, a generalized form of task vectors derived from the historical movements during optimization. Using difference vectors as directed perturbations, we propose the Difference Vector-based Anisotropic Scaling Iterative algorithm (DV-BASI) to enable a continuous optimization process for task arithmetic methods without relying on any additional modules or components. Notably, by leveraging escapability and directional advantages of difference vectors, the average performance on different tasks of the multi-task model merged by DV-BASI may even outperform models individually fine-tuned. Based on this observation, we extend the application of difference vectors to a feasible fine-tuning method for single-task models. On the practical side, DV-BASI allows expressive searching directions with few learnable parameters and forms a scalable framework. We also integrate DV-BASI with task arithmetic methods and advanced optimization techniques to achieve state-of-the-art performance on both supervised and unsupervised evaluation protocols.