Towards Practical Tool Usage for Continually Learning LLMs

TL;DR

This paper investigates how large language models can better adapt to changing environments by leveraging tools and continual learning techniques, demonstrating that scaling models alone is insufficient for adaptation.

Contribution

It introduces a synthetic benchmark and combines NLP tasks to evaluate continual learning in tool-using LLMs, showing that continual learning improves adaptation and reduces forgetting.

Findings

Scaling model size does not improve continual learning.

Tool use combined with continual learning enhances adaptation.

Continual learning reduces forgetting in LLMs.

Abstract

Large language models (LLMs) show an innate skill for solving language based tasks. But insights have suggested an inability to adjust for information or task-solving skills becoming outdated, as their knowledge, stored directly within their parameters, remains static in time. Tool use helps by offloading work to systems that the LLM can access through an interface, but LLMs that use them still must adapt to nonstationary environments for prolonged use, as new tools can emerge and existing tools can change. Nevertheless, tools require less specialized knowledge, therefore we hypothesize they are better suited for continual learning (CL) as they rely less on parametric memory for solving tasks and instead focus on learning when to apply pre-defined tools. To verify this, we develop a synthetic benchmark and follow this by aggregating existing NLP tasks to form a more realistic testing scenario. While we demonstrate scaling model size is not a solution, regardless of tool usage, continual learning techniques can enable tool LLMs to both adapt faster while forgetting less, highlighting their potential as continual learners.