Resistance Training using Prior Bias: toward Unbiased Scene Graph Generation

TL;DR

This paper introduces Resistance Training using Prior Bias (RTPB) and a Dual Transformer (DTrans) backbone to improve scene graph generation, especially for rare relationships, achieving significant performance gains on the VG150 benchmark.

Contribution

The paper proposes a novel bias-based training method and a new encoding backbone to enhance unbiased scene graph generation, addressing data imbalance issues.

Findings

RTPB improves mean recall by over 10%.

DTrans with RTPB outperforms most state-of-the-art methods.

The approach enhances detection of tail categories in scene graphs.

Abstract

Scene Graph Generation (SGG) aims to build a structured representation of a scene using objects and pairwise relationships, which benefits downstream tasks. However, current SGG methods usually suffer from sub-optimal scene graph generation because of the long-tailed distribution of training data. To address this problem, we propose Resistance Training using Prior Bias (RTPB) for the scene graph generation. Specifically, RTPB uses a distributed-based prior bias to improve models' detecting ability on less frequent relationships during training, thus improving the model generalizability on tail categories. In addition, to further explore the contextual information of objects and relationships, we design a contextual encoding backbone network, termed as Dual Transformer (DTrans). We perform extensive experiments on a very popular benchmark, VG150, to demonstrate the effectiveness of our method for the unbiased scene graph generation. In specific, our RTPB achieves an improvement of over 10% under the mean recall when applied to current SGG methods. Furthermore, DTrans with RTPB outperforms nearly all state-of-the-art methods with a large margin.