Skill Neologisms: Towards Skill-based Continual Learning

TL;DR

This paper introduces skill neologisms, soft tokens integrated into language models' vocabularies, enabling scalable, skill-specific learning without weight updates, and demonstrates their composability and effectiveness in natural language tasks.

Contribution

The paper proposes skill neologisms as a novel method for skill-based continual learning, allowing models to acquire and compose new skills without catastrophic forgetting.

Findings

Skill neologisms can be learned to enhance specific skills.

They are composable with out-of-distribution skills.

Zero-shot composition works on the Skill-Mix benchmark.

Abstract

Modern LLMs show mastery over an ever-growing range of skills, as well as the ability to compose them flexibly. However, extending model capabilities to new skills in a scalable manner is an open problem: fine-tuning and parameter-efficient variants risk catastrophic forgetting, while context-based approaches have limited expressiveness and are constrained by the model's effective context. We explore skill neologisms--soft tokens integrated in the model's vocabulary and optimized to improve capabilities over a specific skill--as a way to selectively acquire new skills without weight updates. We first observe that pre-trained LLMs already exhibit tokens associated with procedural knowledge. We then show on a controlled synthetic task that skill neologisms can be learned to improve model capabilities on specific skills while being composable with out-of-distribution skills, and that independently trained skill neologisms can be composed zero-shot. Finally, we validate zero-shot composition of independently learned skill neologisms on the more realistic natural language setting of the Skill-Mix benchmark. These results suggest that skill neologisms may provide a scalable path towards skill-based continual learning.