RAID: A Relation-Augmented Image Descriptor

TL;DR

RAID is a novel image descriptor that captures complex inter-region relations to improve image search based on spatial relationships, outperforming existing methods in identifying images with intricate region interactions.

Contribution

Introduces RAID, a relation-augmented descriptor that models complex inter-region relations for enhanced image retrieval based on spatial relationships.

Findings

Successfully retrieves images with complex relations from COCO dataset

Outperforms existing methods in relation-based image search

Effectively captures spatial distribution of point-to-region relationships

Abstract

As humans, we regularly interpret images based on the relations between image regions. For example, a person riding object X, or a plank bridging two objects. Current methods provide limited support to search for images based on such relations. We present RAID, a relation-augmented image descriptor that supports queries based on inter-region relations. The key idea of our descriptor is to capture the spatial distribution of simple point-to-region relationships to describe more complex relationships between two image regions. We evaluate the proposed descriptor by querying into a large subset of the Microsoft COCO database and successfully extract nontrivial images demonstrating complex inter-region relations, which are easily missed or erroneously classified by existing methods.