MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models

TL;DR

MaZO introduces a novel zeroth-order optimization framework for multi-task fine-tuning of large language models, effectively reducing memory usage and gradient variance, and outperforming existing methods.

Contribution

MaZO is the first framework tailored for multi-task LLM fine-tuning under zeroth-order optimization, addressing gradient variance and task conflict challenges.

Findings

MaZO achieves state-of-the-art multi-task fine-tuning performance.

MaZO surpasses first-order multi-task learning methods.

MaZO reduces memory usage and gradient variance in ZO optimization.

Abstract

Large language models have demonstrated exceptional capabilities across diverse tasks, but their fine-tuning demands significant memory, posing challenges for resource-constrained environments. Zeroth-order (ZO) optimization provides a memory-efficient alternative by eliminating the need for backpropagation. However, ZO optimization suffers from high gradient variance, and prior research has largely focused on single-task learning, leaving its application to multi-task learning unexplored. Multi-task learning is crucial for leveraging shared knowledge across tasks to improve generalization, yet it introduces unique challenges under ZO settings, such as amplified gradient variance and collinearity. In this paper, we present MaZO, the first framework specifically designed for multi-task LLM fine-tuning under ZO optimization. MaZO tackles these challenges at the parameter level through two key innovations: a weight importance metric to identify critical parameters and a multi-task weight update mask to selectively update these parameters, reducing the dimensionality of the parameter space and mitigating task conflicts. Experiments demonstrate that MaZO achieves state-of-the-art performance, surpassing even multi-task learning methods designed for first-order optimization.