Forget Less by Learning from Parents Through Hierarchical Relationships

TL;DR

This paper introduces FLLP, a hierarchical learning framework in hyperbolic space that leverages parent-child relationships to reduce catastrophic forgetting in diffusion models during continual learning.

Contribution

FLLP is the first to embed concept hierarchies in hyperbolic space for continual learning, enhancing knowledge retention and integration of new concepts.

Findings

FLLP improves robustness against forgetting across multiple datasets.

The hyperbolic embedding effectively models hierarchical relationships.

FLLP enhances generalization in continual learning scenarios.

Abstract

Custom Diffusion Models (CDMs) offer impressive capabilities for personalization in generative modeling, yet they remain vulnerable to catastrophic forgetting when learning new concepts sequentially. Existing approaches primarily focus on minimizing interference between concepts, often neglecting the potential for positive inter-concept interactions. In this work, we present Forget Less by Learning from Parents (FLLP), a novel framework that introduces a parent-child inter-concept learning mechanism in hyperbolic space to mitigate forgetting. By embedding concept representations within a Lorentzian manifold, naturally suited to modeling tree-like hierarchies, we define parent-child relationships in which previously learned concepts serve as guidance for adapting to new ones. Our method not only preserves prior knowledge but also supports continual integration of new concepts. We validate FLLP on three public datasets and one synthetic benchmark, showing consistent improvements in both robustness and generalization.