Skew Class-balanced Re-weighting for Unbiased Scene Graph Generation

TL;DR

This paper introduces Skew Class-balanced Re-weighting (SCR), a novel loss function for unbiased scene graph generation that balances predicate prediction performance across long-tailed distributions, improving results on standard datasets.

Contribution

The paper proposes the SCR loss function that re-weights biased predicates to better trade-off between majority and minority predicate performances in SGG.

Findings

SCR improves predicate prediction balance on Visual Genome.

SCR demonstrates generality across multiple SGG models.

Extensive experiments validate SCR's effectiveness.

Abstract

An unbiased scene graph generation (SGG) algorithm referred to as Skew Class-balanced Re-weighting (SCR) is proposed for considering the unbiased predicate prediction caused by the long-tailed distribution. The prior works focus mainly on alleviating the deteriorating performances of the minority predicate predictions, showing drastic dropping recall scores, i.e., losing the majority predicate performances. It has not yet correctly analyzed the trade-off between majority and minority predicate performances in the limited SGG datasets. In this paper, to alleviate the issue, the Skew Class-balanced Re-weighting (SCR) loss function is considered for the unbiased SGG models. Leveraged by the skewness of biased predicate predictions, the SCR estimates the target predicate weight coefficient and then re-weights more to the biased predicates for better trading-off between the majority predicates and the minority ones. Extensive experiments conducted on the standard Visual Genome dataset and Open Image V4 \& V6 show the performances and generality of the SCR with the traditional SGG models.