# On Class Imbalance and Background Filtering in Visual Relationship Detection

## TL;DR

This paper addresses class imbalance and background filtering challenges in Visual Relationship Detection, proposing model and training modifications along with new evaluation measures to improve detection of rare classes and irrelevant relationships.

## Contribution

It introduces novel modifications to VRD models and training procedures, and proposes new evaluation metrics to better assess model performance on class imbalance and background filtering.

## Key findings

- Improved detection of uncommon classes in VRD
- Enhanced filtering of irrelevant background relationships
- More comprehensive evaluation measures for VRD models

## Abstract

In this paper we investigate the problems of class imbalance and irrelevant relationships in Visual Relationship Detection (VRD). State-of-the-art deep VRD models still struggle to predict uncommon classes, limiting their applicability. Moreover, many methods are incapable of properly filtering out background relationships while predicting relevant ones. Although these problems are very apparent, they have both been overlooked so far. We analyse why this is the case and propose modifications to both model and training to alleviate the aforementioned issues, as well as suggesting new measures to complement existing ones and give a more holistic picture of the efficacy of a model.