Taxonomy-Aware Continual Semantic Segmentation in Hyperbolic Spaces for Open-World Perception

TL;DR

This paper introduces TOPICS, a hyperbolic space-based continual semantic segmentation method that leverages taxonomy structures to improve learning of new classes while reducing forgetting in open-world scenarios.

Contribution

The work proposes a novel hyperbolic embedding approach with taxonomy-aware regularization for incremental segmentation, enhancing adaptability and robustness over existing methods.

Findings

Achieves state-of-the-art results on Cityscapes and Mapillary Vistas benchmarks.

Effectively integrates new classes while preserving old class knowledge.

Demonstrates robustness in realistic autonomous driving scenarios.

Abstract

Semantic segmentation models are typically trained on a fixed set of classes, limiting their applicability in open-world scenarios. Class-incremental semantic segmentation aims to update models with emerging new classes while preventing catastrophic forgetting of previously learned ones. However, existing methods impose strict rigidity on old classes, reducing their effectiveness in learning new incremental classes. In this work, we propose Taxonomy-Oriented Poincar\'e-regularized Incremental-Class Segmentation (TOPICS) that learns feature embeddings in hyperbolic space following explicit taxonomy-tree structures. This supervision provides plasticity for old classes, updating ancestors based on new classes while integrating new classes at fitting positions. Additionally, we maintain implicit class relational constraints on the geometric basis of the Poincar\'e ball. This ensures that the latent space can continuously adapt to new constraints while maintaining a robust structure to combat catastrophic forgetting. We also establish eight realistic incremental learning protocols for autonomous driving scenarios, where novel classes can originate from known classes or the background. Extensive evaluations of TOPICS on the Cityscapes and Mapillary Vistas 2.0 benchmarks demonstrate that it achieves state-of-the-art performance. We make the code and trained models publicly available at http://topics.cs.uni-freiburg.de.