HY-WU (Part I): An Extensible Functional Neural Memory Framework and An Instantiation in Text-Guided Image Editing

TL;DR

HY-WU introduces a neural memory framework for continual learning and personalization, enabling instance-specific weight updates without overwriting shared parameters, thus improving adaptability in evolving deployment scenarios.

Contribution

The paper presents HY-WU, a novel functional neural memory framework that synthesizes instance-specific weight updates dynamically, addressing limitations of static weight paradigms in continual learning.

Findings

Enables dynamic, instance-specific weight updates.

Reduces interference and overspecialization in continual learning.

Improves adaptability in evolving deployment environments.

Abstract

Foundation models are transitioning from offline predictors to deployed systems expected to operate over long time horizons. In real deployments, objectives are not fixed: domains drift, user preferences evolve, and new tasks appear after the model has shipped. This elevates continual learning and instant personalization from optional features to core architectural requirements. Yet most adaptation pipelines still follow a static weight paradigm: after training (or after any adaptation step), inference executes a single parameter vector regardless of user intent, domain, or instance-specific constraints. This treats the trained or adapted model as a single point in parameter space. In heterogeneous and continually evolving regimes, distinct objectives can induce separated feasible regions over parameters, forcing any single shared update into compromise, interference, or overspecialization. As a result, continual learning and personalization are often implemented as repeated overwriting of shared weights, risking degradation of previously learned behaviors. We propose HY-WU (Weight Unleashing), a memory-first adaptation framework that shifts adaptation pressure away from overwriting a single shared parameter point. HY-WU implements functional (operator-level) memory as a neural module: a generator that synthesizes weight updates on-the-fly from the instance condition, yielding instance-specific operators without test-time optimization.