Multi-hop Relational Contrastive Learning: Extending Spatial Contrastive Pre-training Beyond Pairwise Relations

TL;DR

This paper introduces MRCL, a novel framework for spatial contrastive learning that captures multi-hop object relations in scene graphs, improving scene understanding and downstream task performance.

Contribution

MRCL extends contrastive learning to multi-hop spatial relations in scene graphs, enabling richer, more robust scene representations beyond pairwise interactions.

Findings

MRCL improves content-based graph retrieval performance (NDCG@5 = 0.748).

MRCL enhances downstream tasks like spatial relationship recognition.

Multi-hop relational supervision yields more robust, compositional, and geometry-aware representations.

Abstract

Understanding how objects relate to each other in space is fundamental to scene understanding, yet most contrastive pre-training approaches only model pairwise relationships, leaving richer compositional and multi-hop interactions largely unexplored. We introduce Multi-Hop Relational Contrastive Learning (MRCL), a framework that extends spatial contrastive learning to graph-structured scene representations. By tracing k-hop paths through scene graphs built from detected objects, MRCL captures implicit spatial dependencies that go well beyond what direct object pairs can express. We define a multi-level contrastive objective spanning nodes, edges, and multi-hop paths, encouraging embeddings that remain stable across object semantics while staying responsive to spatial layout. On a GQA subset, MRCL produces spatially-aware representations that improve content-based graph retrieval (NDCG@5 = 0.748) and consistently benefit downstream tasks, including spatial relationship recognition and graph-based question answering. Together, these results suggest that multi-hop relational supervision offers substantially richer structural guidance than pairwise-only methods, leading to visual representations that are more robust, compositional, and geometry-aware.