Domain-wise Invariant Learning for Panoptic Scene Graph Generation

TL;DR

This paper introduces a novel domain-wise invariant learning framework for Panoptic Scene Graph Generation to mitigate biased predicate annotations, significantly enhancing model performance and generalization in real-world scenarios.

Contribution

The paper proposes a new method to infer and correct biased predicate annotations by learning invariant predicate representations across domains.

Findings

Achieves state-of-the-art performance on PSG benchmarks.

Significantly improves model robustness against biased annotations.

Demonstrates strong generalization on PSG datasets.

Abstract

Panoptic Scene Graph Generation (PSG) involves the detection of objects and the prediction of their corresponding relationships (predicates). However, the presence of biased predicate annotations poses a significant challenge for PSG models, as it hinders their ability to establish a clear decision boundary among different predicates. This issue substantially impedes the practical utility and real-world applicability of PSG models. To address the intrinsic bias above, we propose a novel framework to infer potentially biased annotations by measuring the predicate prediction risks within each subject-object pair (domain), and adaptively transfer the biased annotations to consistent ones by learning invariant predicate representation embeddings. Experiments show that our method significantly improves the performance of benchmark models, achieving a new state-of-the-art performance, and shows great generalization and effectiveness on PSG dataset.