Incremental Human-Object Interaction Detection with Invariant Relation Representation Learning

TL;DR

This paper introduces a novel incremental learning framework for human-object interaction detection that effectively handles continual learning challenges, interaction drift, and zero-shot generalization in dynamic environments.

Contribution

It proposes an exemplar-free incremental relation distillation framework that decouples object and relation learning, improving robustness and generalization in incremental HOI detection.

Findings

Outperforms state-of-the-art methods on HICO-DET and V-COCO datasets.

Effectively mitigates catastrophic forgetting in incremental learning.

Enhances zero-shot HOI detection and robustness against interaction drift.

Abstract

In open-world environments, human-object interactions (HOIs) evolve continuously, challenging conventional closed-world HOI detection models. Inspired by humans' ability to progressively acquire knowledge, we explore incremental HOI detection (IHOID) to develop agents capable of discerning human-object relations in such dynamic environments. This setup confronts not only the common issue of catastrophic forgetting in incremental learning but also distinct challenges posed by interaction drift and detecting zero-shot HOI combinations with sequentially arriving data. Therefore, we propose a novel exemplar-free incremental relation distillation (IRD) framework. IRD decouples the learning of objects and relations, and introduces two unique distillation losses for learning invariant relation features across different HOI combinations that share the same relation. Extensive experiments on HICO-DET and V-COCO datasets demonstrate the superiority of our method over state-of-the-art baselines in mitigating forgetting, strengthening robustness against interaction drift, and generalization on zero-shot HOIs. Code is available at \href{https://github.com/weiyana/ContinualHOI}{this HTTP URL}