Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs

TL;DR

This paper introduces Harmonious Parameter Adaptation (HPA), a novel framework for continual visual instruction tuning of safety-aligned multimodal large language models, effectively balancing safety and task performance while reducing forgetting.

Contribution

HPA is a new post-training method that partitions, selects, and orthogonally adjusts parameters to maintain safety and task accuracy during continual learning.

Findings

HPA outperforms existing methods in safety preservation.

HPA significantly reduces catastrophic forgetting.

HPA achieves balanced safety and task performance.

Abstract

While continual visual instruction tuning (CVIT) has shown promise in adapting multimodal large language models (MLLMs), existing studies predominantly focus on models without safety alignment. This critical oversight ignores the fact that real-world MLLMs inherently require such mechanisms to mitigate potential risks. In this work, we shift our focus to CVIT for safety-aligned MLLMs and observe that during continual adaptation, the model not only suffers from task forgetting but also exhibits degradation in its safety. Achieving a harmonious balance between safety and task performance remains a crucial challenge. To address this, we propose Harmonious Parameter Adaptation (HPA), a post-training framework composed of focusing-based parameter partition, harmoniously balanced parameter selection, and orthogonal parameter adjustment. Specifically, HPA partitions parameters into two types based on their focus on safety or task performance, and selects the focused ones to preserve from a balanced perspective. In addition, HPA imposes orthogonality constraints on parameter updates to further alleviate catastrophic forgetting. Extensive experiments on the CVIT benchmark and safety evaluation datasets demonstrate that HPA better maintains high safety and mitigates forgetting than existing baselines.